noPoll core library manual

Index

Section 1: Basic concepts to starting writing with or integrating noPoll:

• 1.1 How to install noPoll
• 1.2. How to use noPoll library into your application
• 1.3. noPoll thread safety considerations
• 1.4. Creating a noPoll context
• 1.5. Creating a basic WebSocket server with noPoll (using noPoll own loop)
• 1.6. Creating a basic WebSocket client with noPoll
• 1.7. How to handle read I/O operations on noPollConn objects (or how it integrates with existing I/O loops).
• 1.8 Will noPoll automatically integrate fragments into a single message?.

Section 2: Advanced concepts to consider:

• 2.1. Retrying failed write operations
• 2.2. Implementing protocol port sharing: running WebSocket and legacy protocol on the same port

Section 3: using noPoll TLS API:

• 3.1. Implementing mutual TLS certificate verification
• 3.2. Doing extended validation as a post check task (TLS/SSL)
• 3.3. Creating TLS/SSL context to implement especific validation options

Section 4: Android platfom notes:

• 4.1 Android noPoll's usage

1.1 How to install noPoll

Currently, noPoll has only one dependency, which is OpenSSL (libssl) for all those crypto operations required by the protocol itself.

After having that library installed in your system (check your OS documentation), download lastest tar.gz noPoll library from: http://www.aspl.es/nopoll/downloads

Then, to compile the library follow the standard autoconf voodoo:

>> tar xzvf nopoll-{version}.tar.gz
>> cd nopoll-{version}
>> ./configure 
>> make

At this point, if everything went ok, you can check the library by running in one terminal:

>> cd test/
>> ./nopoll-regression-listener

And then in another terminal the client regression test:

>> cd test/
>> ./nopoll-regression-client

Notes about preparing sources if you use SVN/GIT from https://github.com/asples/nopoll

In the case you want to work directly using SVN latest sources, just download them from githubt as usual from: https://github.com/asples/nopoll

After that, run the following command to prepare all compilation files:

>> ./autogen.sh

If everything looks fine, you can install nopoll into your system with the standard:

>> make install

1.2. How to use noPoll library into your application

After a successful noPoll installation, you should be able to include noPoll API functions by including the following header:

#include <nopoll.h>

Then you can use the following to get nopoll compilation flags:

>> pkg-config nopoll --cflags
>> pkg-config nopoll --libs

Or if your project uses autoconf for the building process, just include the following into your configure.ac file:

dnl check for websocket support (through noPoll)
AC_ARG_ENABLE(websocket-support, [  --disable-websocket-support  Makes the built with WebSocket extension library], 
           enable_websocket_support="$enableval", 
           enable_websocket_support=yes)
if test "$enable_websocket_support" != "no" ; then
   PKG_CHECK_MODULES(NOPOLL, nopoll,    [enable_websocket_support=yes], [enable_websocket_support=no])
   AC_SUBST(NOPOLL_CFLAGS)
   AC_SUBST(NOPOLL_LIBS)
fi
AM_CONDITIONAL(ENABLE_WEBSOCKET_SUPPORT, test "x$enable_websocket_support" = "xyes")

1.3. noPoll thread safety considerations

noPoll is designed as a thread agnostic stateless library so it can fit in any project configuration (no matter if it uses threads or event notification).

In the case you are planning to use noPoll in a project that uses threads and you expect to make calls to the noPoll API from different threads at the same time, then you must setup four callbacks that will help noPoll to create, destroy, lock and unlock mutexes. For that, check documentation about nopoll_thread_handlers

1.4. Creating a noPoll context

Before working with noPoll API you must create a noPollCtx object, which represents a single library instance state. You can create as much noPollCtx objects inside the process as you want. To create it you must do something like:

noPollCtx * ctx = nopoll_ctx_new ();
if (! ctx) {
    // error some handling code here
}

// do some WebSocket operations (as client or listener)...

// ...and once you are done and after terminating all messages and
// connections created you have to release the context by doing the
// following:
nopoll_ctx_unref (ctx);

1.5. Creating a basic WebSocket server with noPoll (using noPoll own loop)

Note

Remember you can see many of the code already supported by noPoll by checking the nopoll regression listener at: https://dolphin.aspl.es/svn/publico/nopoll/trunk/test/nopoll-regression-listener.c

Now let's see how to create a simple WebSocket server using noPoll own loop:

// create a listener to receive connections on port 1234
noPollConn * listener = nopoll_listener_new (ctx, "0.0.0.0", "1234");
if (! nopoll_conn_is_ok (listener)) {
     // some error handling here
}

// now set a handler that will be called when a message (fragment or not) is received
nopoll_ctx_set_on_msg (ctx, listener_on_message, NULL);

// now call to wait for the loop to notify events 
nopoll_loop_wait (ctx, 0);

Now, every time a frame is received, the handler listener_on_message will be called. Here is an example about that handler:

void listener_on_message (noPollCtx * ctx, noPollConn * conn, noPollMsg * msg, noPollPtr  user_data) {
        // print the message (for debugging purposes) and reply
        printf ("Listener received (size: %d, ctx refs: %d): (first %d bytes, fragment: %d) '%s'\n", 
                nopoll_msg_get_payload_size (msg),
                nopoll_ctx_ref_count (ctx), shown, nopoll_msg_is_fragment (msg), example);
    
        // reply to the message
        nopoll_conn_send_text (conn, "Message received", 16);
  
        return;
}

1.6. Creating a basic WebSocket client with noPoll

Note

Remember you can see many of the code already supported by noPoll by checking the nopoll regression client at: https://dolphin.aspl.es/svn/publico/nopoll/trunk/test/nopoll-regression-client.c

The process of creating a WebSocket connection is really simple. After creating a context (noPollCtx) you connect to the listener by using:

// call to create a connection 
noPollConn * conn = nopoll_conn_new (ctx, "localhost", "1234", NULL, NULL, NULL, NULL);
if (! nopoll_conn_is_ok (conn)) {
    // some error handling here
}

After that, you can call to nopoll_conn_is_ready to check if the connection is ready to send and receive content. If you don't have a loop to wait on, you can use the following function to wait for the connection to be ready:

if (! nopoll_conn_wait_until_connection_ready (conn, 5)) {
        // some error handling
}

In any case, once the connection is ready, either because nopoll_conn_is_ready returned nopoll_true or because you used nopoll_conn_wait_until_connection_ready, you can send a message by using the following:

// send a message 
if (nopoll_conn_send_text (conn, "Hello there! this is a test", 27) != 27) {
        // send a message
}

Now, to receive the content from this connection see the following.

1.7. How to handle read I/O operations on noPollConn objects (or how it integrates with existing I/O loops).

Now, to receive content from connections (or to handle master listeners requests) you can use the following methods:

• Use noPoll own I/O loop wait (nopoll_loop_wait) and set a onMessage received handler (nopoll_ctx_set_on_msg and nopoll_conn_set_on_msg)
• Use your already working I/O loop to wait for changes on the socket associated to the noPollConn object (see nopoll_conn_socket). In the case of detecting changes on a master listener connection use to nopoll_conn_accept to accept new incoming connections. In the case of detecting changes in connections with I/O, call to nopoll_conn_get_msg to receive entire messages or nopoll_conn_read (if you want to use streaming API).

1.8 Will noPoll automatically integrate fragments into a single message?.

Ok, in general it does, however we would have to look into the particular case to give a correct answer. That's because the term "fragment" is really broad especially for WebSocket.

In general, you shouldn't design in a way that this is important for your application to work. All about WebSocket must be designed as if you were working with a stream oriented socket.

For example, WebSocket standard states that any intermediary may split or join frames. And by any intermediary we mean any WebSocket stack in the middle of the transmission including the sender and the receiver.

The practical implication is that if your WebSocket peer sends a message, it may reach to the other peer consolidated along with other smaller messages or it may be received as several "complete" WebSocket frames.

In general noPoll tries to consolidate internal frame fragments but only for some particular functions like (nopoll_conn_get_msg). For example, if a frame fragment is received, that is, just the header and part of the body, this particular function will "wait" until the entire body is received. And by "wait" we mean the function will return NULL, waiting you to call again in the future to get the entire message.

In general, even though the standard allows it (as we saw), noPoll doesn't do any split or join operation to avoid confusion. However, some API calls like nopoll_conn_read, which provides an "stream view" of the connection, will serve bytes as they come (so the frame concept is not present in this case).

2.1. Retrying failed write operations

Every time you do a write operation (using for example nopoll_conn_send_text or nopoll_conn_send_text_fragment) there is a possibility that the write operation failes because the socket isn't capable to keep on accepting more data.

In that case, errno == 11 (or NOPOLL_EWOULDBLOCK) is returned so you can check this to later retry the write operation.

Because websocket involves sending headers that already includes the size of the message sent, you can't just retry by calling again to the send operation used (like nopoll_conn_send_text), especially because you must "continue" the send operation where it was left instead of sending more content with additional headers. In short, you must complete the operation.

To this end, you must use the following functions to check and complete pending write operations:

• nopoll_conn_pending_write_bytes
• nopoll_conn_complete_pending_write

Here is a posible complete function considering all points:

int websocket_write (noPollConn * conn, const char * content, int length) {
          // FIRST PART: normal send operation
          int tries = 0;
          int bytes_written;

          // do write operation and check
          bytes_written = nopoll_conn_send_text (conn, content, length);
          if (bytes_written == length) {
                 // operation completed, just return bytes written
                 return bytes_written;
          } 

          // SECOND PART: retry in the case of failure
          // some failure found, check errno
          while (tries < 5 && errno == NOPOLL_EWOULDBLOCK && nopoll_conn_pending_write_bytes (conn) > 0) {
                 // ok, unable to write all data but that data is waiting to be flushed
                 // you can return here and then make your application to retry again or
                 // try it right now, but with a little pause before continue
                 nopoll_sleep (10000); // lets wait 10ns

                 // flush and check if write operation completed
                 if (nopoll_conn_complete_pending_write (conn) == 0)
                         return length;
                  
                 // limit loop
                 tries++;
          }
        
          // failure, return error code reported by the first call or the last retry
          return  bytes_written;
}

As we can see, the example tries to first write the content and then check for errors, trying to complete write in the case of errno == NOPOLL_EWOULDBLOCK, but, before going ahead retrying, the function sleeps a bit.

A very important note to consider is that this isn't by far the best way to do this. This example is just to demonstrate the concept. The "ideal" implementation would be not to do any retry here (second part) but let the engine looping and waiting for this WebSocket to retry later, letting the overall application to keep on doing other things meanwhile (like writing or handling I/O in other connections) rather than locking the caller (as the example do).

Knowing this, if you want a ready to use function that implements concept (for the second part), you can directly use:

• nopoll_conn_flush_writes

With it, a fairly complete and efficient write operation would be:

// do write operation 
bytes_written = nopoll_conn_send_text (conn, content, length);

// complete pending write by flushing and limitting operation for 2 seconds
// pass to the function bytes_written as returned by nopoll_conn_send_text
bytes_written = nopoll_conn_flush_writes (conn, 2000000, bytes_written);

2.2. Implementing protocol port sharing: running WebSocket and legacy protocol on the same port

Current noPoll design allows to implement full WebSocket connection accept by calling to nopoll_conn_accept but also it is possible to let other application to do the accept connection, try to guess if what is comming is a WebSocket connection, to then let noPoll to complete the accept socket operation.

As a example, these concepts are being implemented by Vortex Library (http://www.aspl.es/vortex) to allow running on the same port BEEP and BEEP over WebSocket.

The way each application implements "port sharing concept" is really especific to each case, but here are the general steps:

• 1. Let the legacy application to accept the socket by the standard call accept ()
• 2. Then, call to recv (socket, buffer[3], 3, MSG_PEEK); to get just 3 bytes from the socket without removing them from the queue.
• 3. Then check with something like the following to know if the incoming connection seems to be a WebSocket one or not:

// detect tls conn 
nopoll_bool is_tls_conn = bytes[0] == 22 && bytes[1] == 3 && bytes[2] == 1;

// detect then both values (TLS WebSocket and just WebScoket)
if (! axl_memcmp ("GET", bytes, 3) && ! is_tls_conn)
        return nopoll_false; // nothing detected here (it doesn't seems
                        // to be a websocket connection) continue as normal

// nice, it seems we've found an incoming WebSocket connection

• 4. In the case nothing was detected, stop, and continue with the usual accept process that was already in place. In the case you detected a posible WebSocket connection, then, you'll have to do something like:

// Create a noPollConn listener object that presents the legacy listener where the connection was
// received. In general is recommended to reuse these references to avoid creating over and over
// again new references as new WebSocket connections are received.
//
noPollConn * nopoll_listener = nopoll_listener_from_socket (nopoll_ctx, listener_socket);

// Create an accepted listener connection reusing the socket received
// where nopoll_ctx is a reference to a noPollCtx object created and reused by all
// connections accepted, and _socket represents the socket file descriptor
//
noPollConn * conn = nopoll_listener_from_socket (nopoll_ctx, _socket);
if (! nopoll_conn_is_ok (conn)) {
      // ok, something failed, release and return
      nopoll_conn_close (conn);
      return;
}

// Now, initiate the entire WebSocket accept process (including TLS one)
// where nopoll_ctx is the context we used before, nopoll_listener is a listener where
// the connection was received, conn is the connection accepted and is_tls_conn
// is an indication about what to expect about TLS process.
//
if (! nopoll_conn_accept_complete (nopoll_ctx, nopoll_listener, conn, _socket, is_tls_conn)) {
      // failed to accept connection, release connection
      nopoll_conn_close (conn);
      // optionally close listener reference if it is not reused
      nopoll_conn_close (nopoll_listener);
      return;
}

// now process incoming messages as configured (either because you've configured an onMessage handler)
// or because you are handling directly all incoming content (streaming API).

3.1. Implementing mutual TLS certificate verification

In the case you want to verify provided client certificate at the server you can use the following code. It optionally provides the CA (as root.pem) which will make the listener to also verify client matches that CA. However, if you remove that CA (root.pem) your server will just verify peer's certificate:

// configure server certificates (server.pem) signed by the
// provided ca (root.pem) also configured in the last
// parameter 
if (! nopoll_conn_opts_set_ssl_certs (opts, 
                                      "server.pem",
                                      "server.key",
                                      NULL,
                                      "root.pem")) {
       printf ("ERROR: unable to setup certificates...\n");
       return -1;
}

// configure peer verification 
nopoll_conn_opts_ssl_peer_verify (opts, nopoll_true);

// create listener         
listener2 = nopoll_listener_tls_new_opts (ctx, opts, "0.0.0.0", "1239");
if (! nopoll_conn_is_ok (listener2)) {
    printf ("ERROR: Expected to find proper listener TLS connection status (:1236, SSLv23), but found..\n");
    return -1;
} 

Here the important part is calling to nopoll_conn_opts_ssl_peer_verify. This is because peer verification is disabled for servers (that is, servers by default do not verify peer's certificate which usually is not provided). However, this is not true for clients which is by default enabled (of course, client must always verify server's certificate when connecting with TLS/SSL, was we have said, this can be controlled by nopoll_conn_opts_ssl_peer_verify too).

Now, in the case you want to provide client certificate for a WebSocket connecting node, use the following:

// create connection options 
opts     = nopoll_conn_opts_new ();
nopoll_conn_opts_set_ssl_certs (opts, 
                           // certificate 
                           "client.pem",
                           // private key 
                           "client.pem",
                           NULL,
                           // ca certificate 
                           "root.pem");

// connect to remote WebSocket server
conn = nopoll_conn_tls_new (ctx, opts, "server-address", "1239", NULL, NULL, NULL, NULL);

3.2. Doing extended validation as a post check task (TLS/SSL)

The idea is to configure the following handler at your server/listener component:

nopoll_ctx_set_post_ssl_check (ctx, your_post_ssl_check_handler, <some-user-pointer>);

Now, somewhere, that handler will have the following signature and code example:

nopoll_bool  your_post_ssl_check_handler (noPollCtx      * ctx,
                                          noPollConn     * conn,
                                          noPollPtr        SSL_CTX,
                                          noPollPtr        SSL,
                                          noPollPtr        user_data)
{
       
      // Do here some additional checks on the certificate received (using SSL_CTX and SSL). 
      // I the case of error, return nopoll_false to ensure the connection is not accepted. 
           
      return nopoll_true; // to accept connection 
}

For example, to get the certificate that is proposing the client you could use:

cert = SSL_get_peer_certificate (SSL);

3.3. Creating TLS/SSL context to implement especific validation options

In the case you want to create the SSL_CTX/SSL object so it uses certain configurations like chain certificates, etc, so that the verification will only work when matching these settings.

If this is the case you'll have to use a different handler that will help noPoll engine to create the SSL context with the settings you want. By default, the SSL context is created with default settings if no handler is provided.

In the case you want to provide a ssl context creator, use:

nopoll_ctx_set_ssl_context_creator (ctx, your_ssl_context_creator, NULL);

And your handler will have the following signature with a very basic configuration:

SSL_CTX * your_ssl_context_creator (noPollCtx * ctx, noPollConn * conn, noPollConnOpts * opts, nopoll_bool is_client, noPollPtr user_data)
{
    // very basic context creation using default settings provided by OpenSSL
    return SSL_CTX_new (is_client ? TLSv1_client_method () : TLSv1_server_method ()); 
}

As you can see, the function must return an SSL_CTX for every connection received and attempting to start TLS session.