This section discusses the semantic meaning and layout of protocol messages. For details on how Neo4j types are represented in binary form, see serialization.
Clients may send request messages at any time after a session is initialized. Clients may pipeline requests, sending multiple requests together.
Servers must fully respond to each request before the next request is processed and processing of requests within a session must be done in the same order in which the requests.
Servers must ignore messages sent by the client after a failure occurs on the server, until the client has acknowledged the failure. See Failure & Acknowledgement.
For each request message sent, clients must anticipate receiving zero or more detail messages followed by exactly one summary message.
The detail messages deliver the response content, while a summary message denotes the end of the response and any response metadata.
Note that "detail" and "summary" are classifications of message, not specific message types.
For example, RECORD
messages are classed as detail messages and SUCCESS
messages as summary messages.
The diagrams below illustrates a basic exchange wherein the client sends a request message and receives a series of response messages.
Initialization
Before a session can be used to run queries, it must be initialized.
The INIT
message should be sent by the client as the first message it sends after negotiating protocol version.
Pipelining
The client is not required to wait for a response before sending more messages. Sending multiple messages together like this is called pipelining:
For performance reasons, it is recommended that clients use pipelining as much as possible. Through pipelining, multiple messages can be transmitted together in the same network package, significantly reducing latency and increasing throughput.
Tip A common technique is to buffer outgoing messages on the client until the last possible moment, such as when a commit is issued or a result is read by the application, and then sending all messages in the buffer together. |
Failure handling
Because the protocol leverages pipelining, the client and the server need to agree on what happens when a failure occurs, otherwise messages that were sent assuming no failure would occur might have unintended effects.
When requests fail on the server, the server will send the client a FAILURE
message.
The client must acknowledge the FAILURE
message by sending an RESET
message to the server.
Until the server receives the RESET
message, it will send an IGNORED
message in response to any other message from by the client, including messages that were sent in a pipeline.
The RESET
clears the pending failure state, disposes of any outstanding records and rolls back the current transaction (if any).
The diagram below illustrates a typical flow involving RESET
messages:
Here, the original failure is acknowledged immediately by the client, allowing the subsequent RUN to be actioned as expected.
This second diagram shows a sequence where a pair of request messages are sent together:
Here, the client optimistically sends a pair of messages. The first of these fails and the second is consequently IGNORED
.
Once the client acknowledges the failure, it is then able to resend a corrected RUN message.
Messages
Protocol messages are represented as serialized structures.
INIT
The INIT
message is a client message used once to initialize the session.
This message is always the first message the client sends after negotiating protocol version.
Sending any message other than INIT
as the first message to the server will result in a FAILURE
and the
server closing the connection.
All parameters in the INIT
message are required.
Response
-
SUCCESS {}
if initialization has completed successfully -
FAILURE {"code": ..., "message": ...}
if the request was malformed, or if initialization cannot be performed at this time, or if the authorization failed.
InitMessage (signature=0x01) { String clientName Map<String,Value> authToken }
Example
Value: INIT "MyClient/1.0" { "scheme": "basic", "principal": "neo4j", "credentials": "secret"} B1 01 8C 4D 79 43 6C 69 65 6E 74 2F 31 2E 30 A3 86 73 63 68 65 6D 65 85 62 61 73 69 63 89 70 72 69 6E 63 69 70 61 6C 85 6E 65 6F 34 6A 8B 63 72 65 64 65 6E 74 69 61 6C 73 86 73 65 63 72 65 74
Initialize message parameters
Parameter | Description |
---|---|
clientName | A name and version for the client, if the user has allowed usage data collection, this is used to track popular clients. Example: "MyClient/1.0" |
authToken | An authorization token used to authenticate to the database. The token must contain either just the entry { "scheme" : "none" } or the keys scheme, principal, and credentials. Example { "scheme": "basic", "principal": "neo4j", "credentials": "secret"}". If no scheme is provided, it defaults to "none". |
RUN
The RUN
message is a client message used to pass a Cypher statement for execution on the server. It has the following structure:
RunMessage (signature=0x10) { String statement Map<String,Value> parameters }
On receipt of a RUN
message, the server will start a new job by executing the statement with the parameters supplied.
If successful, the subsequent response will consist of a single SUCCESS
message; if not, a FAILURE
response will be sent instead.
A successful job will always produce a result stream which must then be explicitly consumed (via PULL_ALL
or DISCARD_ALL
), even if empty.
Depending on the statement you are executing, additional metadata may be returned in both the SUCCESS
message from the RUN
, as well as in the final SUCCESS
after the stream has been consumed.
It is up to the statement you are running to determine what meta data to return.
Notably, most queries will contain a fields metadata section in the SUCCESS
message for the RUN
statement, which lists the result record field names.
We list further examples of meta data in the examples section.
In the case where a previous result stream has not yet been fully consumed, an attempt to RUN
a new job will trigger a FAILURE
response.
If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single
IGNORED
message and take no further action.
Response
-
SUCCESS {"fields": ...}
if the statement has been accepted for execution -
FAILURE {"code": ..., "message": ...}
if the request was malformed or if a statement may not be executed at this time
Example
Value: RUN "RETURN 1 AS num" {} B2 10 8F 52 45 54 55 52 4E 20 31 20 41 53 20 6E 75 6D A0
DISCARD_ALL
The DISCARD_ALL
message is a client message used to discard all remaining items from the active result
stream. It has the following structure:
DiscardAllMessage (signature=0x2F) { }
On receipt of a DISCARD_ALL
message, the server will dispose of all remaining items from the active result stream, close the stream and send a single SUCCESS
message to the client.
If no result stream is currently active, the server will respond with a single FAILURE
message.
If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single IGNORED
message and take no further action.
Response
-
SUCCESS {}
if the result stream has been successfully discarded -
FAILURE {"code": ..., "message": ...}
if no result stream is currently available
Example
Value: DISCARD_ALL B0 2F
PULL_ALL
The PULL_ALL
message is a client message used to retrieve all remaining items from the active result stream.
It has the following structure:
PullAllMessage (signature=0x3F) { }
On receipt of a PULL_ALL
message, the server will send all remaining result data items to the client, each in a single RECORD
message.
The server will then close the stream and send a single SUCCESS
message optionally containing summary information on the data items sent.
If an error is encountered, the server must instead send a FAILURE
message, discard all remaining data items and close the stream.
If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single IGNORED
message and take no further action.
Response
-
SUCCESS {...}
if the result stream has been successfully transferred -
FAILURE {"code": ..., "message": ...}
if no result stream is currently available or if retrieval fails
Example
Value: PULL_ALL B0 3F
RESET
The RESET
message is a client message used to return the current session to a "clean" state.
It will cause the session to IGNORE
any message it is currently processing, as well as any message before RESET
that had not yet begun processing.
This allows RESET
to abort long-running operations.
It also means clients must be careful about pipelining RESET
.
Only send this if you are not currently waiting for a result from a prior message, or if you want to explicitly abort any prior message.
The following actions are performed by RESET
:
-
force any currently processing message to abort with
IGNORE
-
force any pending messages that have not yet started processing to be
IGNORED
-
clear any outstanding
FAILURE
state - dispose of any outstanding result records
- rollback the current transaction (if any)
See the section called “Error handling with RESET” for example usage.
Also, see the section called “ACK_FAILURE” for a message that only clears FAILURE
state
ResetMessage (signature=0x0F) { }
Response
-
SUCCESS {}
if the session was successfully reset -
FAILURE {"code": ..., "message": ...}
if a reset is not currently possible
Example
Value: RESET B0 0F
ACK_FAILURE
The ACK_FAILURE
message is a client message used to acknowledge a failure the server has sent.
It is similar to RESET
, but it does not roll back open transactions, nor does it interrupt running operations.
This can be a better option than RESET
in cases where a client wants to explicitly call "ROLLBACK" in case of failure.
A good example of this is in a shell environment, where an error should cause all subsequent statements to fail until the transaction is rolled back.
The following actions are performed by ACK_FAILURE
:
-
clear any outstanding
FAILURE
state - dispose of any outstanding result records
See the section called “Error handling with ACK_FAILURE” for an example.
AckFailureMessage (signature=0x0E) { }
Response
-
SUCCESS {}
if the session was successfully reset -
FAILURE {"code": ..., "message": ...}
if a reset is not currently possible
Example
Value: ACK_FAILURE B0 0E
RECORD
The RECORD
message is a server detail message used to deliver data from the server to the client.
Each record message contains a single List, which in turn contains the fields of the record in order.
It has the following structure:
RecordMessage (signature=0x71) { List<Value> fields }
Example
Value: RECORD [1,2,3] B1 71 93 01 02 03
SUCCESS
The SUCCESS
message is a server summary message used to signal that a corresponding client message has been received and actioned as intended.
The message contains a map of metadata, the contents of which depend on the original request.
It has the following structure:
SuccessMessage (signature=0x70) { Map<String,Value> metadata }
Example
Value: SUCCESS { "fields": ["name", "age"]} B1 70 A1 86 66 69 65 6C 64 73 92 84 6E 61 6D 65 83 61 67 65
FAILURE
The FAILURE
message is a server summary message used to signal that a corresponding client message has encountered an error while being processed.
It has the following structure:
FailureMessage (signature=0x7F) { Map<String,Value> metadata }
FAILURE
messages contain metadata providing details regarding the primary failure that has occurred.
This metadata is a simple map containing a code and a message. These codes map to the standard Neo4j status codes.
When a FAILURE
occurs, in most cases any open transaction will be rolled back.
However, if the FAILURE
is classified as a client error
, the transaction will be left open and can be used again
after the FAILURE
has been acknowledged.
This is mainly to support user-driven queries, where a database administrator may have built up a large transaction, and
we do not want a simple spelling mistake to roll it all back.
Example
Value: FAILURE { "code": "Neo.ClientError.Statement.SyntaxError", "message": "Invalid syntax." } B1 7F A2 84 63 6F 64 65 D0 25 4E 65 6F 2E 43 6C 69 65 6E 74 45 72 72 6F 72 2E 53 74 61 74 65 6D 65 6E 74 2E 53 79 6E 74 61 78 45 72 72 6F 72 87 6D 65 73 73 61 67 65 8F 49 6E 76 61 6C 69 64 20 73 79 6E 74 61 78 2E
IGNORED
The IGNORED
message is a server summary message used to signal that a corresponding client message has been ignored and not actioned.
It has the following structure:
IgnoredMessage (signature=0x7E) { Map<String,Value> metadata }
A client message will be ignored if an earlier failure has not yet been acknowledged by the client via a RESET
message.
For example, this will occur if the client optimistically sends a group of messages, one of which fails during execution: all subsequent messages in that group will then be ignored.
Note that the original PULL_ALL
message was never processed by the server.
Example
Value: IGNORED B0 7E