Peer Protocol Introduction
--------------------------

The peer protocol has been designed primarily for use with CIYAM blockchain applications (limited interactive
testing via the "ciyam_client" is also supported to assist with development) and its small number of commands
is just one of its features to reduce potential issues that might arise from "bad actors".

Because of the many potential attack vectors that any P2P protocol presents it's not recommended for features
(especially any new command) to be added without considerable testing and review from other developers.

Protocol Commands
-----------------

The protocol commands can be divided into those for connection/disconnection ("tls" and "bye") and those that
are used for the general peer to peer communication ("chk", "get", "msg" and "put").

The "tls" command is optional and can only be issued immediately after the peer has connected. If issued then
a TLS session will be commenced (i.e. all further communications will be performed over a secure socket). The
"bye" command is also optional and is a formal way of informing a peer that a session has finished.

The "chk" command allows a peer to inquire as to whether another peer has a particular file or the content of
a file (the latter being via an optional "nonce" argument). After commencing a session the "chk" command must
be the first command to be issued. It should be noted that unless doing interactive testing any "chk" command
that has provided a "tag" rather than a file "hash" in its request must provide a tag value which is prefixed
with the blockchain prefix literal (i.e. "@blockchain"). This command can be issued at any time by peers.

One could imagine a peer accepting files from other peers but then not keeping them stored for any reasonable
amount of time so normally a peer will check that another peer has retained the content of a previously "put"
file using the "chk" command and its "nonce" argument.

The "get" command is used in order to fetch blockchain and arbitrary files during peer synchronisation. It is
the content of specific blockchain files that determines which arbitrary files will then later be fetched.

The "msg" command allows peers to exchange arbitrary messages which by convention will start with the content
type following by a ':' separator and then the message content itself (e.g. "utf8:test"). Being arbitrary the
messages may be simply ignored by peers and are not expected to be persisted (unlike arbitrary files).

It should be noted that a "put" file is not an arbitrary file but instead will contain public keys and hashes
for one or more arbitrary files that had been locally encrypted previously.

Interactive Testing
-------------------

In order to interactively test the CIYAM peer protocol a "-test_peer_port=<port>" option can be used when the
application server is being started. If needing to test for some time (such as when developing a new protocol
command) a special configuration option can be added to "ciyam_server.sio" (which is found immediately before
the "<default_storage>" entry):

 ./ciyam_server -test_peer_port=11111 &

[or alternatively edit "ciyam_server.sio"]
...
 <test_peer_port>11111
# <default_storage>test
...

After providing the startup option (or changing the configuration file and restarting the application server)
the "ciyam_server.log" should look similar to the following:

[2020-04-07 17:52:25] [000000] [general] server started on port 11011 (pid = 888)
[2020-04-07 17:52:25] [000000] [general] peer listener started on port 11111

To connect use "ciyam_client" specifying the port number (i.e. 11111 per above) as follows:

./ciyam_client 11111

To see a list of all the available peer commands after starting the client type "?" and hit enter:

> ?
commands:
=========
chk <tag_or_hash> [<nonce>] (check if peer has a file or hash the content of a file with a nonce)
get <tag_or_hash> [<name>] (fetch a file from the peer files area or blockchain archive)
put <hash> (send a file with public key and hash information for repository processing)
msg <data> (exchange an arbitrary data message)
tls (start TLS session)
bye (terminate session)

It should be noted that unlike the standard application server protocol the peer protocol uses a finite state
machine to determine what commands are permitted to be executed at any particular point in time (the "?" help
command can be executed at any time but is only permitted to be executed once per session).

The following illustrates a simple interactive peer session:

./ciyam_client 11111

> chk hello
(not found)
put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
get cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> ~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> get cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
msg utf8:test

> msg utf8:test
utf8:test
get cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> ~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> bye

Testing General Peer File Storage
---------------------------------

Interactive testing does not use a blockchain and is only useful for testing the peer protocol commands. This
does not provide a very useful understanding of how actual blockchain peers synchronise but is useful to form
a basic understanding about repository entry handling (being a key part of the file backup security).

When an arbitrary file has been retrieved it is immediately encrypted by the peer that issued the "get" (thus
the original file content itself is not persisted in order to ensure that there is no risk of storing content
that might be regarded as illegal). The private key that was used to generate the "shared secret" is not kept
by the peer but the public key is. This means that the peer cannot unencrypt the content but the owner of the
content can decrypt this encrypted content provided they are given the public key that was used by the peer.

These sample four interactive sessions (two standard protocol and two peer protocol sessions) illustrates how
an arbitrary file can be stored by a peer and then later retrieved by its "owner". For this first session the
owner of a file creates a repository storage request (which will be later used via the peer "put" command):

#-----------------------------------------------------------------------------------------------------------#
# NOTE: Copy and paste the following into your terminal session:                                            #
#-----------------------------------------------------------------------------------------------------------#
 ./ciyam_server -quiet -sid=9a96178dfe775b4baaeed2488c44a54d -test_peer_port=11111 &
sleep 2
 ./ciyam_client -quiet
file_put test.jpg
peer_persist_file a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753 dummy
file_get a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
file_get 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
file_kill a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
file_kill 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
quit

#------------------------------------------------------------------------------------------------------------#
# Output from the above should be as follows:                                                                #
#------------------------------------------------------------------------------------------------------------#

> file_put test.jpg

> peer_persist_file a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753 dummy
52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282

> file_get a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> file_get 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282

> file_kill a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> file_kill 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282

> quit

The final argument in the "peer_persist_file" command (i.e. "dummy") is the password for key pair generation.
The hash "a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753" is the content hash of "test.jpg"
whilst "52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282" is the file repository push request
hash with its content being as follows:

md:raw
pk:Aj4XW1jZbIENUUW8Q0dWWoEPoWliHngyNgvZDe+Eyu9o
sh:pascJuUlP7cxa1Hn9AaHGDcU4NaDA0lU4ej8Z7ykJ1M=

Next a peer session is started in order to send this file repository storage request via a "put" (a "get" for
the hash "a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753" will occur after the "put" that's
issued for "52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282"):

#-----------------------------------------------------------------------------------------------------------#
# NOTE: Copy and paste the following into your terminal session:                                            #
#-----------------------------------------------------------------------------------------------------------#
 ./ciyam_client -quiet 11111
chk hello
put 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
~rm 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
~rm a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
bye

#------------------------------------------------------------------------------------------------------------#
# Output from the above should be as follows:                                                                #
#------------------------------------------------------------------------------------------------------------#

> chk hello
(not found)
put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> put 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
get a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> ~rm 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282

> ~rm a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> ~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> bye

After this the encrypted file content exists in the file system and the file repository contains an entry for
tying the original content hash to the locally encrypted content. If using an "interactive test session" then
the hash of the encrypted file will be b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5 which
in base64 format is "tpYzrBqEdIQE+vsQvPYFYCMui82P2k8FcLn5BLaW8qU=".

So let's assume the peer that initially stored the repository file ("test.jpg") no longer possesses a copy of
that file but does know the original file hash and the password for the master private key. In order to fetch
an encrypted copy from a peer's repository they would initially perform a "chk" using the original file hash.

The other peer would then perform a "put" of a file that contains the repository pull request information for
the requested file that they will report as being "not found" (as they only have an encrypted version of it).
This "raw" file contains a public key (that is required in order to decrypt) along with the "source hash" (of
the encrypted content) and the "target hash" (being the hash of the originally pushed content).

#-----------------------------------------------------------------------------------------------------------#
# NOTE: Copy and paste the following into your terminal session:                                            #
#-----------------------------------------------------------------------------------------------------------#
 ./ciyam_client -quiet 11111
chk hello
chk a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
~cat 08a3a00aa1a55dce8c42ec576e667b7af43f9d2cdaa01c8c1055b46acded1eac
~rm 08a3a00aa1a55dce8c42ec576e667b7af43f9d2cdaa01c8c1055b46acded1eac
put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
get b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5
~rm b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5
~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
bye

#------------------------------------------------------------------------------------------------------------#
# Output from the above should be as follows:                                                                #
#------------------------------------------------------------------------------------------------------------#

> chk hello
(not found)
put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> chk a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
(not found)
put 08a3a00aa1a55dce8c42ec576e667b7af43f9d2cdaa01c8c1055b46acded1eac

> ~cat 08a3a00aa1a55dce8c42ec576e667b7af43f9d2cdaa01c8c1055b46acded1eac
md:raw
pk:Aj4XW1jZbIENUUW8Q0dWWoEPoWliHngyNgvZDe+Eyu9o
sh:tpYzrBqEdIQE+vsQvPYFYCMui82P2k8FcLn5BLaW8qU=
th:pascJuUlP7cxa1Hn9AaHGDcU4NaDA0lU4ej8Z7ykJ1M=
> ~rm 08a3a00aa1a55dce8c42ec576e667b7af43f9d2cdaa01c8c1055b46acded1eac

> put cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef
get cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> get b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5
msg utf8:test

> ~rm b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5

> ~rm cceeb7a985ecc3dabcb4c8f666cd637f16f008e3c963db6aa6f83a7b288c54ef

> bye

A final standard protocol session can now decrypt and extract the original file content to "xxx.jpg" follows:

#-----------------------------------------------------------------------------------------------------------#
# NOTE: Copy and paste the following into your terminal session:                                            #
#-----------------------------------------------------------------------------------------------------------#
 ./ciyam_client -quiet
file_tag b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5\
 @a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
file_tag b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5\
 ~023e175b58d96c810d5145bc4347565a810fa169621e7832360bd90def84caef68
peer_persist_file -dh=a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753\
 b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5 dummy
file_get a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753 xxx.jpg
~diff xxx.jpg test.jpg
~rm xxx.jpg
file_kill a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753
file_kill 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282
storage_init Meta
pd admin @now Meta Global_Repository_Entry\
 .pascJuUlP7cxa1Hn9AaHGDcU4NaDA0lU4ej8Z7ykJ1M.tpYzrBqEdIQE-vsQvPYFYCMui82P2k8FcLn5BLaW8qU
quit

#-----------------------------------------------------------------------------------------------------------#
# Output from the above should be as follows:                                                               #
#-----------------------------------------------------------------------------------------------------------#

> file_tag b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5\
 @a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> file_tag b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5\
 ~023e175b58d96c810d5145bc4347565a810fa169621e7832360bd90def84caef68

> peer_persist_file -dh=a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753\
 b69633ac1a84748404fafb10bcf60560232e8bcd8fda4f0570b9f904b696f2a5 dummy

> file_get a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753 xxx.jpg

> ~diff xxx.jpg test.jpg

> ~rm xxx.jpg

> file_kill a5ab1c26e5253fb7316b51e7f40687183714e0d683034954e1e8fc67bca42753

> file_kill 52326e48bde0bf6bb78fe965728c8679a5594e42d9cb0f95b9df41b391903282

> storage_init Meta

> pd admin @now Meta Global_Repository_Entry\
 .pascJuUlP7cxa1Hn9AaHGDcU4NaDA0lU4ej8Z7ykJ1M.tpYzrBqEdIQE-vsQvPYFYCMui82P2k8FcLn5BLaW8qU

> quit