3.1 Connecting two peers across a NAT

A common use is to let two programs on two different machines talk straight to each other, even when one or both sit behind a home router that blocks incoming connections.

1. Each side binds an endpoint with make-endpoint. It uses iroh’s default relays, so it can be reached from outside its own network.
2. Each side reads its own address with endpoint-address. The address holds a node id, which names the peer, together with routing hints: the relay URL and any direct socket addresses the peer found for itself.
3. The two sides swap their addresses. guile-iroh does not do this step for you. Send the address over any channel that both machines can already reach or by copying it across by hand.
4. One side dials the other with endpoint-connect, passing the address it received. The other side waits for the call with endpoint-accept. The connection is checked against the node id, so a wrong or altered routing hint cannot hand you a different peer.
5. iroh does the NAT traversal, the connection is started through the relay, and it switches to a direct link between the two machines once the two sides manage to reach each other.
6. Once connected, one side opens a bidirectional stream with connection-open-stream and the other takes it with connection-accept-stream. Each returns two values: a send half and a receive half.
7. Now the two sides can exchange bytes. send-write puts the bytes of a bytevector onto the stream, and recv-read returns the next bytes off it as a bytevector (or the end-of-file object at end of stream). Both calls block until they are done. Since the stream is bidirectional, both sides can write and read at the same time. When a side has nothing more to send it calls send-finish, which marks the stream finished without waiting for the bytes to reach the peer.

Every call here blocks the thread that makes it, so a real program should usually drive them from its own event loop or from lightweight threads. see Driving guile-iroh from Fibers does the latter.

The two programs below exemplify the whole workflow. listen.scm prints its own address as a ticket (a single string carrying the node id and routing hints) then serves peers one at a time, echoing back the first message of each:

;; listen.scm
(use-modules (iroh)
             (ice-9 receive)
             (rnrs bytevectors))

(define endpoint (make-endpoint "guile-iroh/example/0"))

;; Print our address as a ticket, for dial.scm's command line.
(display (address->string (endpoint-address endpoint)))
(newline)
(force-output)

;; Serve one peer after another, until interrupted.
(let loop ()
  (let ((connection (endpoint-accept endpoint)))
    (when connection
      (receive (send recv) (connection-accept-stream connection)
        (let ((message (recv-read recv 1024)))
          (send-write send message)      ;echo the message back
          (send-finish send)))
      (loop))))

send-finish returns as soon as the stream is marked finished, before the echo is known to have arrived.

dial.scm takes the printed ticket on its command line, connects, sends one message and reads the echo:

;; dial.scm
(use-modules (iroh)
             (ice-9 receive)
             (rnrs bytevectors))

(define address (string->address (cadr (command-line))))

(define endpoint (make-endpoint "guile-iroh/example/0"))

(define connection (endpoint-connect endpoint address))
(receive (send recv) (connection-open-stream connection)
  (send-write send (string->utf8 "hello over iroh"))
  (send-finish send)
  (let ((echo (recv-read recv 1024)))
    (format #t "echoed back: ~a~%" (utf8->string echo))))

Start the listener on one machine and copy its output line, the ticket. It goes on serving until you interrupt it with C-c:

$ guile -L . listen.scm
endpointaa…

Then dial from the other machine, pasting that ticket as the argument:

$ guile -L . dial.scm endpointaa…
echoed back: hello over iroh

Both invocations expect the library to be locatable, see Locating the library at run time and guile-iroh to be available to Guile. Note that the two endpoints must use the same protocol string (the first argument of make-endpoint), or the dial is refused.

3.2 Driving guile-iroh from Fibers

The programs of the previous section do one thing at a time. A program that has to dial several peers, or serve several of them, usually does so with some kind of concurrency. A popular Guile library is Fibers. This example shows two Fibers program interacting with each other over iroh.

A fiber is not a thread: it shares a scheduler thread with the other fibers, and it gives that thread up only at a suspension point. Since a guile-iroh call is a blocking foreign call, it blocks the scheduler thread for its whole duration. So, for example, a fiber calling endpoint-accept freezes every other fiber that scheduler owns.

The way out is to keep the blocking call off the scheduler thread altogether: run it in a POSIX thread, and have the fiber wait for its result on a channel. This works because a channel operation may be performed by a thread that is not a fiber, in which case it blocks that thread rather than suspending a fiber. offload.scm implements this like so:

;; offload.scm
(define-module (offload)
  #:use-module (fibers channels)
  #:use-module (ice-9 match)
  #:use-module (ice-9 threads)
  #:export (in-thread))

(define (in-thread thunk)
  "Call THUNK in a fresh POSIX thread and return its values.  Only the calling
fiber waits: the scheduler thread stays free to run the other fibers."
  (define channel (make-channel))
  (call-with-new-thread
   (lambda ()
     ;; A channel may be used from a thread that is not a fiber: the put
     ;; blocks this thread until a fiber takes the message.
     (put-message channel
                  (with-exception-handler
                    (lambda (exception)
                      (cons 'raise exception))
                    (lambda ()
                      (cons 'return (call-with-values thunk list)))
                    #:unwind? #t))))
  (match (get-message channel)
    (('return . vals) (apply values vals))
    (('raise . exception) (raise-exception exception))))

With it, the listener from the previous example turns into a server that gives every peer its own fiber.

;; listen-fibers.scm
(use-modules (iroh)
             (offload)
             (fibers)
             (ice-9 receive)
             (rnrs bytevectors))

(define endpoint (make-endpoint "guile-iroh/example/0"))

(display (address->string (endpoint-address endpoint)))
(newline)
(force-output)

(define (echo connection)
  (receive (send recv) (in-thread (lambda () (connection-accept-stream connection)))
    (let ((message (in-thread (lambda () (recv-read recv 1024)))))
      (in-thread (lambda () (send-write send message)))
      (in-thread (lambda () (send-finish send))))))

(run-fibers
 (lambda ()
   (let loop ()
     (let ((connection (in-thread (lambda () (endpoint-accept endpoint)))))
       (when connection
         ;; Serve this peer in its own fiber and go straight back to
         ;; accepting: slow peers do not hold up the next one.
         (spawn-fiber (lambda () (echo connection)))
         (loop))))))

The dialer, in turn, can dial every peer at once. dial-fibers.scm takes one ticket per peer on its command line, dials them in parallel and prints the echoes as they land:

;; dial-fibers.scm
(use-modules (iroh)
             (offload)
             (fibers)
             (fibers channels)
             (ice-9 match)
             (ice-9 receive)
             (rnrs bytevectors))

(define tickets (cdr (command-line)))

(define endpoint (make-endpoint "guile-iroh/example/0"))

(define (echo-through address message)
  "Dial ADDRESS, send MESSAGE and return the echo, as a string."
  (let ((connection (in-thread (lambda () (endpoint-connect endpoint address)))))
    (receive (send recv) (in-thread (lambda () (connection-open-stream connection)))
      (in-thread (lambda () (send-write send (string->utf8 message))))
      (in-thread (lambda () (send-finish send)))
      (utf8->string (in-thread (lambda () (recv-read recv 1024)))))))

(run-fibers
 (lambda ()
   (define echoes (make-channel))
   ;; One fiber per ticket: every peer is dialed at the same time, however
   ;; long the NAT traversal of any one of them takes.
   (for-each (lambda (ticket)
               (spawn-fiber
                (lambda ()
                  (let ((address (string->address ticket)))
                    (put-message echoes
                                 (cons (address-node-id address)
                                       (echo-through address "hello over iroh")))))))
             tickets)
   ;; Print the echoes as they arrive, in whatever order they do.
   (for-each (lambda _
               (match (get-message echoes)
                 ((node-id . echo)
                  (format #t "~a echoed back: ~a~%"
                          (substring node-id 0 8) echo))))
             tickets)))

One endpoint serves all the fibers of a program: iroh dials and accepts through a shared reference, so concurrent endpoint-connect and endpoint-accept calls on the same endpoint are fine. A connection and the two halves of a stream, on the other hand, are each meant for one fiber at a time.

Run the two programs the way you ran the previous pair, from a directory where Guile can find both guile-iroh and offload.scm. Start one listener per machine:

$ guile -L . listen-fibers.scm
endpointaa…

Then dial them all from a third, passing one ticket per peer:

$ guile -L . dial-fibers.scm endpointaa… endpointab…
0f8a3d21 echoed back: hello over iroh
b4c07e95 echoed back: hello over iroh

The echoes are printed in the order they arrive rather than in ticket order. Both programs speak the same protocol as the blocking ones, so dial.scm can dial listen-fibers.scm and dial-fibers.scm can dial listen.scm.

The above code is just an example, a production setup requires more attention. in-thread starts one thread per call, which is cheap next to a NAT traversal but not free. A program that moves many small messages is better served by keeping a thread per connection (one thread taking requests off a channel and running them in order) rather than by spawning a new one for every read and write.

3.3 Locating the library at run time

When developing, guile-iroh resolves the shared library through the GUILE_IROH_SOFILE_DEV environment variable (in production it falls back to the library directory configured at build time, if you installed guile-iroh from your distro). The value is the path of libiroh_c_ffi.so without the .so extension. When running your own programs against an uninstalled checkout, export it:

export GUILE_IROH_SOFILE_DEV="/path/to/iroh-c-ffi/target/release/libiroh_c_ffi"

With the variable set, run the examples of the previous sections from the top of the checkout:

$ guile -L . listen.scm

4.1 (iroh endpoint)

• Procedures

4.2 (iroh connection)

• Procedures

4.3 (iroh address)

• Procedures