Thingstream Client Library  BLD4131-v2.13
Thingstream Client Library Documentation

Thingstream Client SDK

The Thingstream Client SDK provides functionality for devices to connect to the Thingstream servers and exchange messages with other devices on a Thingstream domain.

Getting started

This Arduino release of the Thingstream Client SDK contains the following example applications to help you get started:

Thingstream client SDK builds include libraries in binary format for a specific target, as well as an example serial driver and main program. The example sources are for illustrative purposes only and are not intended for use in a production system.

System architecture

The client communicates with the server over the MQTT-SN protocol. Please refer to the MQTT-SN specification for an explanation on topics, QOS levels and the connection lifecycle.

Some MQTT-SN features are not needed for Thingstream or not yet implemented:

  • Gateway advertisement and discovery is not needed
  • Wills are not currently supported

Useful documents:

Architecture

A typical application using the Thingstream Client SDK will mainly refer to the client API functions (client_api.h). The client uses an implementation of the transport API (transport_api.h) to send and receive data.

Transport API implementations are stackable, with each layer providing encapsulation as required by the underlying transport media.

Application
Client (client_api.h)
Transport (transport_api.h)
[Transport]
...
Driver (transport_api.h)

The dual protocol USSD/UDP modem transport (modem2_transport.h) is described below. If you are using the original USSD-only modem transport (modem_ussd_transport.h) then the Thingstream stack setup and Buffer usage is described here.

The required stack for communicating with Thingstream infrastructure looks like this

Application
Client (client_api.h)
Protocol layer (thingstream_transport.h)
Base64 codec (base64_codec_transport.h) (optional for UDP)
* Modem transport (modem2_transport.h)
* Ring buffer input (ring_buffer_transport.h)
* Serial port driver (serial_transport.h)

The layers marked '*' depend on the porting code for the target.

This architecture allows adding additional transport layers, for example to provide log output during development. While testing layers have been implemented, they are not yet part of the SDK, due to platform dependencies.

Example code initializing the stack

Note: when calling Thingstream_createModem2Transport() the Thingstream_uBloxSaraR4Init parameter (one of many choices) must match the particular modem hardware when using UDP; when using USSD this parameter should be Thingstream_UssdInit.

// create the lowest level transport (note that the arguments to
// serial_transport_create(...) are target specific and are defined
// in the target specific file serial_transport.h)
ThingstreamTransport* transport = serial_transport_create("/dev/ttyS0");
ASSERT(transport != NULL);
// If the serial transport receives bytes directly from an interrupt service
// routine then a ring buffer transport is required.
// The buffer must be statically allocated.
transport = Thingstream_createRingBufferTransport(transport, ring_buf, sizeof(ring_buf));
ASSERT(transport != NULL);
// wrap modem to/from serial activity with logger if required
if (logflags != 0)
{
transport = Thingstream_createModemLogger(transport, printf, logflags);
ASSERT(transport != NULL);
}
// wrap it into a modem transport
// The buffer must be statically allocated and is used to buffer data sent
// over UDP or USSD.
// When using UDP we suggest a buffer size of MODEM2_UDP_BUFFER_LEN (approx 1 Kb);
// when using just USSD, use a buffer size of MODEM2_USSD_BUFFER_LEN.
transport = Thingstream_createModem2Transport(transport, 0,
modem_buf, sizeof(modem_buf),
NULL);
ASSERT(transport != NULL);
// wrap it for base64 encoding (optional for when using UDP)
ASSERT(transport != NULL);
// wrap Thingstream protocol implementation
transport = Thingstream_createProtocolTransport(transport, NULL, 0);
ASSERT(transport != NULL);
// wrap client to/from thingstream activity with logger if required
if (logflags != 0)
{
transport = Thingstream_createClientLogger(transport, printf, logflags);
ASSERT(transport != NULL);
}
// create the client utilizing the stack.
ASSERT(client != NULL);
// initialise the client before use
ASSERT(cRes == CLIENT_SUCCESS);

Buffers

In order to minimize the memory requirement of the client stack, a shared buffer is used for sending and receiving data. This buffer should be passed to the modem transport implementation and is passed up via the ThingstreamTransport::get_buffer() function to higher levels in the transport stack.

If the serial transport receives bytes directly from an interrupt service routine then an additional ring buffer is required. If the serial transport obtains bytes by polling (inside the serial->run() routine) then no ring buffer is required.

The Thingstream_createProtocolTransport() function can optionally be passed a buffer which is used to store larger messages (when using USSD "larger" messages are 76 bytes or more, when using UDP (without base64) and a modem_buffer of MODEM2_UDP_BUFFER_LEN then "larger" messages are 977 bytes or more).

Control flow

Client APIs that send data will block until the data is sent and appropriately acknowledged by the server. Internally, the Client will call the underlying ThingstreamTransport::run() method, which trickles all the way down to the lowest level Transport implementation, where any time waiting for incoming data is spent.

The implementation of the lowest level transport is platform specific. Its ThingstreamTransport::run() method may choose to enter a low-power state until woken by an interrupt indicating incoming data. This is also the best place to refresh any configured watchdogs.

While the application is connected to the server (i.e. has successfully done a Thingstream_Client_connect() but not yet done a Thingstream_Client_disconnect()) the application is expected to periodically call Thingstream_Client_run() in order to process any incoming data. If there are incoming MQTT-SN messages then they will be passed to Thingstream_Application_subscribeCallback() where the application can process the data of the message.

Restrictions using Thingstream Callbacks

Note that all of the Thingstream_Application_xxxxCallback() functions must observe the following restrictions:

  • It is not permitted to call other Thingstream_Client_xxx() apis from within callbacks
  • Any data passed to the callback may be overwritten as soon as the callback returns
  • The callbacks will only ever be called from inside one of:

Platform porting APIs

Hardware platforms other than the Thingstream reference platforms must at least supply their own implementations of the serial port driver and the functions in client_platform.h.

Other modules are platform independent.

However, a user may decide to implement a custom modem implementation by replacing the included modem_transport (and perhaps also serial transport) with their own transport implementation.

Implementing the serial driver

The serial transport module is target specific and, on hardware platforms other than the Thingstream reference platforms, needs to be implemented by the user.

If the serial transport receives bytes directly from an interrupt service routine then an additional ring buffer is required. If the serial transport obtains bytes by polling (inside the serial_run() routine) then no ring buffer is required.

It is expected that the serial driver implemention will either synchronously implement ThingstreamTransport::send() or copy the data into an independent buffer for asynchronous sending. The maximum length of the serial ThingstreamTransport::send() data is typically less than 1100 bytes. If asynchronous sending is implemented then the serial transport must have a mechanism to either cancel any previously active send or to wait until it has completed.

The serial transport's ThingstreamTransport::run() function may return prematurely if no data is available, but we recommend that it performs a wait-for-interupt before returning.
The Thingstream_Client_xxx() apis will repeatedly call the ThingstreamTransport::run() function until an expected response is received or a client level timeout occurs.

The innermost transport ThingstreamTransport::run() method is a good place for refreshing watchdog timers or performing other critical tasks.

The serial driver's ThingstreamTransport::get_client_id() is not currently used in a full Thingstream MQTT-SN stack.


Release Notes

Release History

Release v2.13: 18-Aug-2021

  • SDK: restart waiting-for-ACK timeout after send
  • Add 'const' to type of last arg in Thingstream_Util_parseXxx()
  • Add Thingstream_Util_run* functions
  • SDK: update for the UDP/USSD dual protocol transport (modem2)
    • u-blox: SARA-R4: remove AT+UFOTACONF=2 from init sequence
    • u-blox: issue AT+USOER if socket open fails
    • Pass all input to Thingstream_Application_modemCallback()
    • u-blox: SARA-U2: fix init by ignoring CEREG errors
  • SDK: only issue 'clear FPLMN' if list is not already empty
  • Improve behaviour of Thingstream_Util_printf()

Release v2.12: 29-Jun-2021

  • SDK: update for the UDP/USSD dual protocol transport (modem2)
    • Ignore errors from diagnostic commands
    • u-blox: SARA-R4: wait for 'AcT' before choosing 2G/LTE commands
    • u-blox: SARA-R4: implement fallback from AT+UCGED=5 to =2

Release v2.11: 21-Jun-2021

  • SDK: update for the UDP/USSD dual protocol transport (modem2)
    • u-blox SARA-R4: change init to work for both R41x and R422

Release v2.9: 11-Jun-2021

  • SDK: updates for the UDP/USSD dual protocol transport (modem2)
  • SDK: send ACK before returning from Thingstream Client_XXX() APIs

Release v2.8: 25-May-2021

  • SDK: new file thingstream_result.h with ThingstreamClientResult and ThingstreamTransportResult definitions
  • Update transport version check in all the examples
  • SDK: use SDK constants to control modem init timeout and retry count
  • SDK: updates for the UDP/USSD dual protocol transport (modem2)
    • Sim7080: change shutdown AT+CNCLOSE=1 to AT+CACLOSE=1
    • u-blox: SARA-R4: show operator when no local IP from AT+CGDCONT
    • u-blox: SARA-R4: retry send() if +USOER:11
    • u-blox: SARA-R4: use different modem init sequence for 2G and LTE
    • u-blox: SARA-R4: improve diagnostics by reordering and adding more AT commands
    • u-blox: restrict +CGACT: detection to cid 1
    • u-blox: improve robustness by resetting state during initialisation
    • Add Thingstream_uBloxTobyL2Init support
    • Allow Thingstream_Modem2_sendLine() to be called before Thingstream_Client_init()

Release v2.7: 11-Feb-2021

  • SDK: updates for the UDP/USSD dual protocol transport (modem2)
    • u-blox SARA-R5: deactivate the PDP context in Thingstream_Client_shutdown()
    • u-blox SARA-R5: change modem init sequence (wait for +CGREG/+CEREG)
    • u-blox SARA-R4: change modem init sequence (wait for +CGREG/+CEREG and IP addr)

Release v2.6: 22-Jan-2021

  • SDK: updates for the UDP/USSD dual protocol transport (modem2)
    • u-blox modems: fix handling of pending inbound data
    • u-blox SARA-R4: add workaround for loss of PDP context
  • Add example for Nordic nRF52840
  • SDK: Fix delayed ACK after PUBACK
  • Update main Thingstream Client SDK documentation page to presume modem2
  • SDK: Retire deregister_callback transport entry point

Release v2.5: 29-Oct-2020

Release v2.4: 13-Aug-2020

Release v2.3: 18-Jun-2020

Release v2.2: 20-Feb-2020

Release v2.1: 09-Jan-2020

  • SDK: Add UDP config for Quectel BG96 and UG96
  • Provide debug_printf_core.c as source
  • SDK: retire deregister_callback transport entry point.
  • SDK: Tidyup support for KEIL and RVCT
  • SDK: Fix some ublox modem2 problems (including buffer corruption)
  • Docs: update header file documentation
  • SDK: escape backslash in modem logger transport

Release v2.00: 05-Nov-2019

Release v1.43: 10-Sep-2019

  • SDK: Improve portability of gcc compiled sdk objects
  • SDK: Fix thingstream_transport_create() when called with NULL buffer
  • SDK: Support +CREG:5,XXXX,XXXX (without quotes)

Release v1.40: 12-Aug-2019

  • Docs: Update icon

Release v1.39: 01-Aug-2019

Release v1.38: 22-Jul-2019

  • SDK: Add new block types to thingstream log transport

Release v1.37: 03-Jul-2019

  • SDK: Change Topic type definition to fix alignment issue
  • SDK: Change to use Generic as default user agent
  • SDK: Fix documentation of Client_subscribeName() and Client_subscribeTopic()
  • SDK: Treat topic names of 1 or 2 bytes as topicTypeShort
  • SDK: Fix client logging of UNSUBACK and UNSUBSCRIBE

Release v1.36: 24-May-2019

Release v1.35: 25-Apr-2019

  • SDK: Change Client_destroy() to flush and shutdown transport
  • SDK: Fix 'waiting for ACK' when publishing at QoS0/QoS-1

Release v1.34: 11-Apr-2019

  • SDK: Build gcc arm SDK objects with neutral wchar_t and FPU attributes
  • SDK: Rename internal base64_xxx() functions to avoid name-clash
  • SDK: Client_run() wait of zero is now 'smallest time' rather than 100ms minimum
  • SDK: Restrict length of USSD line sent to modem (THINGSTREAM_USSD_BUFFER_LEN)

Release v1.33: 05-Mar-2019

  • Docs: Add 'table of contents' to documentation pages
  • SDK: Add cortex A7 builds

Release v1.32: 22-Feb-2019

  • SDK: Fix DEBUG=1 assertion in DISCONNECT or PINGREQ

Release v1.31: 07-Feb-2019

Release v1.30: 18-Jan-2019

  • SDK: Change handling of errors during inbound messages
  • SDK: Change Copyright notices to Thingstream AG
  • SDK: Make AT&W in init sequence optional
  • SDK: fix retry logic for publish QoS2
  • SDK: Make modem->shutdown() call inner->shutdown()
  • SDK: Protect loggers against undefined callbacks

Release v1.29: 06-Nov-2018

  • SDK: Fix a Thingstream layer delay
  • SDK: Work-around Quectel modem firmware issue with sequence packets

Release v1.27: 08-Oct-2018

  • SDK: check correct Transport instance passed to Modem_xxx routines e.g. Modem_set_modem_callback()
  • SDK: Fix Client_run() which could miss inbound messages.
  • SDK: Only issue AT+CUSD=2 after AT+CUSD=1 during init.
  • SDK: Fix parsing +CREG: S,"LLLL","CCCC" response

Release v1.26: 10-Sep-2018

Release v1.25: 23-Aug-2018

  • SDK: Fix description of PredefinedSelfTopic
  • SDK: Add TLOG_TIME logging level for log_xxx_transport_create(), to replace previous action of TLOG_VERBOSE
  • SDK: Add logger for thingstream <=> base64 layers
  • SDK: Enhance parsing of +CREG:
  • SDK: Modem driver: Add "\r\n" to lines that do not end in "\n"
  • SDK: Protect Client_xxx() against NULL client pointers

Release v1.24: 13-Jul-2018

  • SDK: Optionally ignore inbound +CUSD:2 (required by some modems)
  • SDK: Return CLIENT_NOT_CONNECTED if server disconnects

Release v1.23: 29-Jun-2018

  • SDK: Fix inbound QoS2 handling
  • SDK: log_client_transport remove duplicate output

Release v1.19: 10-May-2018

  • SDK: Fix bug with thingstream-layer transport with no buffer.
  • SDK: Fix modem after AT+CFUN=1,1 recovery.
  • SDK: Treat Client_publish() @ QoS-1 with named topic as error
  • SDK: We have found that inbound publish at QoS2 is not handled correctly. This issue will be fixed in a later release.

Release v1.18: 10-Apr-2018

  • SDK: log_client_transport show CONNECT flags

Release v1.17: 02-Apr-2018

  • SDK: Increase number of line buffer transports to 3

Release v1.14: 16-Mar-2018

  • SDK: Fix checksum calculation for 8-bit MCUs

Release v1.12: 09-Mar-2018

  • SDK: Allow Client_run() with zero timeout
  • SDK: Fix modem skip() parser and reduce its code size

Release v1.11: 27-Feb-2018

  • SDK: Fix modem initialization string (must have a AT+CREG?)
  • SDK: Wait for server response before returning from Client_disconnect()
  • SDK: Configure modem to return verbose error messages (AT+CMEE=2)

Release v1.10: 20-Feb-2018

Release v1.03: 15-Feb-2018

  • SDK: Add support for server-controlled USSD end-session

Release v1.02: 24-Jan-2018

  • SDK: Check for NULL callback in thingstream_layer.c

Release v1.01: 21-Dec-2017

  • Modem: Fix race in OK wait logic
  • Modem: Reject unexpected +CUSD message
  • Modem: Make USSD end session switchable split or merged
  • Modem: Fix modem wait for OK after AT+CUSD=2

Release v1.00: 11-Dec-2017

  • Initial Client SDK release

ThingstreamClient
struct ThingstreamClient_s ThingstreamClient
Definition: client_api.h:50
Thingstream_uBloxSaraR4Init
ThingstreamModem2UdpInit Thingstream_uBloxSaraR4Init
Thingstream_createClientLogger
ThingstreamTransport * Thingstream_createClientLogger(ThingstreamTransport *inner, ThingstreamPrintf_t log, uint8_t level_mask)
Thingstream_createBase64CodecTransport
ThingstreamTransport * Thingstream_createBase64CodecTransport(ThingstreamTransport *inner)
Thingstream_createModem2Transport
ThingstreamTransport * Thingstream_createModem2Transport(ThingstreamTransport *inner, uint16_t flags, uint8_t *buffer, uint16_t bufSize, ThingstreamModem2UdpInit udpConfigInit, ThingstreamPrintf_t logger)
Thingstream_createProtocolTransport
ThingstreamTransport * Thingstream_createProtocolTransport(ThingstreamTransport *inner, uint8_t *buffer, uint16_t len)
ThingstreamResult
ThingstreamResult
Definition: thingstream_result.h:34
Thingstream_createClient
ThingstreamClient * Thingstream_createClient(ThingstreamTransport *transport)
CLIENT_SUCCESS
@ CLIENT_SUCCESS
Definition: thingstream_result.h:36
ThingstreamTransport
Definition: transport_api.h:147
Thingstream_createRingBufferTransport
ThingstreamTransport * Thingstream_createRingBufferTransport(ThingstreamTransport *inner, uint8_t *data, uint16_t size)
Thingstream_Client_init
ThingstreamClientResult Thingstream_Client_init(ThingstreamClient *client)
Thingstream_createModemLogger
ThingstreamTransport * Thingstream_createModemLogger(ThingstreamTransport *inner, ThingstreamPrintf_t log, uint8_t level_mask)