Robot API for Autonomous Machines

Overview

The NeuroSynthetica Sentience Engine™ allows external computer systems, called robots, to connect as clients, and engage the server through its I/O channels. Through this mechanism, robots may transmit sensory data to the server, and perform behavioral functions modulated by data received from the server.

Licensees and community members working with the Sentience Engine can create robot software applications that run on Linux hosts scaling from ARM-based Raspberry Pi devices up to X64-based PC platforms. Robot applications call the NeuroSynthetica Robot API, which is exposed by the NeuroSynthetica Robot Library, available to all NeuroSynthetica members via the community portal.

Throughout this discussion, the formal term MISI is used to refer to the Sentience Engine's Model Input Signal Interface; it is simply the technical name for the input channel. Similarly, the formal term MOSI is used to refer to the Sentience Engine's Model Output Signal Interface, the technical name for the output channel.

Robot Software Operation

In general, robot applications call the Robot Library's API functions to supply their credentials and indicate which simulations they are willing to connect to. Simulations may be specified by name, or by Simulation ID, or both. Then, they enter a polling loop, where they continuously wait for Sentience Engines to load the specified simulation. Because the servers may be stopped and restarted by other clients, such as NeuroSynthetica Workbench management consoles, robot applications may continuously scan for the servers to become available. As they do, input and output channels become available and can be opened by the robot application.

Robots need not interact purely with the physical world using physical sensors and actuators; they may also include virtual sensors, such as data that may be available from the Internet, or they may operate entirely virtually, without any physical embodiment at all. For example, a robot might run as a Linux service on a VM and navigate web sites on the Internet.

Once input channels are opened, they may be stimulated by the robot application by calling RobotSendMisiDatum(), RobotSendMisiIndexedPoint(), RobotSendMisiDataVector(), RobotSendMisiDataMatrix(), RobotSendMisiDataBitmap(), or RobotSendMisiDataVolume API functions.

When output channels are opened, the robot application registers a callback routine which is invoked by the Robot Library when data associated with the output channel arrive. The callback routine may perform whatever behavioral actions might be appropriate in the callback routine, including activities such as initiating the actuating of servos or changing the volume, frequency, or timbre of a vocalization.

Robot Library

The following is a summary of API functions supported by the NeuroSynthetica Robot library. This list is intended to provide a sense of how it works, but is not intended to be API documentation or a definitive list; that information is available to subscribers on the Community Portal.

SPARK_STATUS RobotInitialize ();

SPARK_STATUS RobotAddHost (IN char *HostName);

UINT64 RobotMillisecondTimeStamp (void);

SPARK_STATUS RobotSpecifyServerCredentials (IN char *Username, IN char *Password);

SPARK_STATUS RobotSpecifySimulationId (IN UINT64 SimulationId);

SPARK_STATUS RobotSpecifySimulationName (IN char *SimulationName);

SPARK_STATUS RobotStartSimulation ();

SPARK_STATUS RobotStopSimulation ();

SPARK_STATUS RobotOpenMisiChannel (IN char *ChannelName, OUT int *ChannelIndex);

SPARK_STATUS RobotQueryMisiChannelDimensions (
    IN int ChannelIndex,
    OUT int *NumberOfDimensions,
    OUT PUINT16 Extents);

SPARK_STATUS RobotStimulateMisiChannel (IN int ChannelIndex);

SPARK_STATUS RobotSendMisiDatum (
    IN int ChannelIndex,
    IN UINT8 ScalarValue);

SPARK_STATUS RobotSendMisiIndexedPoint (
    IN int ChannelIndex,
    IN PUINT16 RelativeCoordinates,
    IN UINT8 ScalarValue);

SPARK_STATUS RobotSendMisiDataVector (
    IN int ChannelIndex,
    IN PUINT16 RelativeCoordinates,
    IN UINT16 XExtent,
    IN PUINT8 LinearArray);

SPARK_STATUS RobotSendMisiDataMatrix (
    IN int ChannelIndex,
    IN PUINT16 RelativeCoordinates,
    IN UINT16 XExtent,
    IN UINT16 YExtent,
    IN PUINT8 Matrix);

SPARK_STATUS RobotSendMisiDataMatrixBitmap (
    IN int ChannelIndex,
    IN PUINT16 RelativeCoordinates,
    IN UINT16 XExtent,
    IN UINT16 YExtent,
    IN PUINT8 Matrix);

SPARK_STATUS RobotSendMisiDataVolume (
    IN int ChannelIndex,
    IN PUINT16 RelativeCoordinates,
    IN UINT16 XExtent,
    IN UINT16 YExtent,
    IN UINT16 ZExtent,
    IN PUINT8 Volume);

SPARK_STATUS RobotOpenMosiChannel (IN char *ChannelName, OUT int *ChannelIndex);

SPARK_STATUS RobotRegisterMosiActivationRtn (
    IN int ChannelIndex,
    IN void (*ActivationRtn)(IN UINT64 Context, IN int NumberOfDimensions, IN PUINT16 Coordinates, IN int Psp),
    IN UINT64 Context);

Sample Robot Source Code

The following example Hello World robot application is written in C. It demonstrates how NeuroSynthetica Robot API functions are called to find a Sentience Engine running a simulation, connecting to input and output channels, and interacting with the I/O channels.

//
// MODULE NAME.
//      Hello.c - "Hello World" Robot.
//
// FUNCTIONAL DESCRIPTION.
//      This module implements a simple robot module for to test 
//      linkage to the server set and to demonstrate how to implement MISI 
//      (model input) and MOSI (model output) on the robot.
//
//      To use this:
//      1. Build the Robot Library, then build this module.
//      2. Create a user 'robot' with password 'cyberdyne' using uadmin on the server.
//      3. Create a project with a MISI node "Input Value" and a MOSI node 
//         "Output Value" on the server, using WB's model definition language.
//      4. Close the simulation with the WB and also close the project, to save everything.
//      5. Reopen the project, load, and start the simulation with WB.
//      6. Run ./Hello on the robot connected to the same network.
//         a) By default, the robot will listen to ANY and ALL simulations, so if 
//         you have several going, use -s "simulation-name" on the command line
//         to specify the simulation name you're using in WB.
//         b) Specify the IPv4 host addresses of the Sentience Servers hosting
//         the I/O channels to be connected to, with the '-h' switch.
//         c) By default, the robot list login to the server with username "robot"
//         and password "cyberdyne". If you selected different credentials with uadmin,
//         then specify them with -u username and -p password on the command line.
//
//      7. The robot will engage in protocol with the server, and error messages
//         and progress messages will be displayed as the robot finds the resources
//         on the server that it needs. It is NORMAL for it to not find its MISI
//         and MOSI channels for a few seconds as it enumerates them in parallel.
//         Then, this chatter should go away, and you should only see messages from
//         MosiActivationRtn(), in this module. These messages prove that the MOSI
//         channel "Output Value" is streaming its data from the server to the
//         robot routine. Obviously this is an example to show how to make all of
//         that happen; from here, you can clone this whole robot module and start
//         creating a module that does something else with these values, such as
//         moving legs or motors, or making noise, etc.
//
// MODIFICATION HISTORY.
//      S. E. Jones     20/09/10.       #1.0, new.
//      S. E. Jones     20/10/29.       #1.0, added narrative above.
//      S. E. Jones     21/01/11.       #1.0, added MISI channel.
//
// NOTICE.
//      Copyright (C) 2020-2021, NeuroSynthetica, LLC. All Rights Reserved.
//

//
// Standard includes.
//

#include <inttypes.h>
#include <math.h>

//
// NeuroSynthetica includes.
//

#include "../../h/Spark.h"
#include "../../h/Robot.h"

//
// Private constant and type declarations.
//

#define MOSI_CONTEXT_OUTPUT_VALUE       1

//
// Forward routine declarations.
//

static void StimulateMisiChannel ();
static void MosiActivationRtn (IN UINT64 Context, IN int NumberOfDimensions, IN PUINT16 Coordinates, IN int Psp);
static void Help ();

//
// Data exposed by this module for use by other modules.
//

//
// External routine declarations (exposed by other modules).
//

//
// Private static data local to this module.
//

static int MisiInputValueChannel = -1;
static int MosiOutputValueChannel = -1;

static UINT64 FirstTimeStamp = 0;
static UINT64 NextMisiTimeStamp = 0;
static float MisiValue = 0.0;

static char *CommandLineSimulationName = NULL;
static char *CommandLineUsername = "robot";
static char *CommandLinePassword = "cyberdyne";

#define MAX_HOSTS 20

static int NumberOfCommandLineHosts = 0;
static char *CommandLineHosts [MAX_HOSTS];

//
// Global data referenced by this module.
//

//
// Module API routine implementations (exposed to other modules).
//

//
// FUNCTION NAME.
//      main - Main Entrypoint.
//
// FUNCTIONAL DESCRIPTION.
//      This function is called by the C library to begin execution of
//      the robot module.
//
// MODIFICATION HISTORY.
//      S. E. Jones     20/09/10.       #1.0, new.
//      S. E. Jones     21/02/11.       #1.0, call robot lib for ms timestamp.
//
// ENTRY PARAMETERS.
//      argc            - number of arguments passed in argv array.
//      argv            - array of pointers to ASCIIZ strings representing arguments.
//
//  EXIT PARAMETERS.
//      Function Return - Linux status code.
//

int main (
    IN int argc, 
    IN char **argv
    )
{
    SPARK_STATUS rc;
    int i;
    char *p;

    fprintf (stderr, "Hello World Robot Module\n");

    for (i=1; i<argc; i++) {
        p = argv [i];
        if (!strcmp (p, "-s")) {
            if (i < argc+1) {
                CommandLineSimulationName = argv [i+1];
                i++;
                continue;
            }
            fprintf (stderr, "main: Missing simulation name.\n");
            Help ();
            exit (-98);
        }
        
        if (!strcmp (p, "-u")) {
            if (i < argc+1) {
                CommandLineUsername = argv [i+1];
                i++;
                continue;
            }
            fprintf (stderr, "main: Missing username.\n");
            Help ();
            exit (-97);
        }
        
        if (!strcmp (p, "-p")) {
            if (i < argc+1) {
                CommandLinePassword = argv [i+1];
                i++;
                continue;
            }
            fprintf (stderr, "main: Missing password.\n");
            Help ();
            exit (-96);
        }

        if (!strcmp (p, "-h")) {
            if (i < argc+1) {
                if (NumberOfCommandLineHosts >= MAX_HOSTS) {
                    fprintf (stderr, "main: Too many hosts; limit %d.\n", MAX_HOSTS);
                    exit (-94);
                }
                CommandLineHosts [NumberOfCommandLineHosts++] = argv [i+1];
                i++;
                continue;
            }
            fprintf (stderr, "main: Missing host.\n");
            Help ();
            exit (-95);
        }

        if (!strcmp (p, "--help")) {
            Help ();
            exit (0);
        }

        fprintf (stderr, "main: invalid command line option '%s'.\n", p);
        Help ();
        exit (-99);
    }

    //
    // Initialize the Robot Library (launches network thread, etc.)
    //

    rc = RobotInitialize ();
    if (rc != SPARK_STATUS_SUCCESS) {
        fprintf (stderr, "main: RobotInitialize failed, rc=%d.\n", rc);
        exit (-1);
    }
               
    for (i=0; i<NumberOfCommandLineHosts; i++) {
        rc = RobotAddHost (CommandLineHosts [i]);
        if (rc != SPARK_STATUS_SUCCESS) {
            fprintf (stderr, "main: RobotAddHost(%s) failed, rc=%d.\n", CommandLineHosts [i], rc);
            exit (-1);
        }
    }

    //
    // Supply user credentials from the command line.
    //

    rc = RobotSpecifyServerCredentials (CommandLineUsername, CommandLinePassword);
    if (rc != SPARK_STATUS_SUCCESS) {
        fprintf (stderr, "main: RobotSpecifyServerCredentials() failed, rc=%d.\n", rc);
        exit (-2);
    }

    //
    // Set the simulation name from the command line.
    //

    if (CommandLineSimulationName != NULL) {
        rc = RobotSpecifySimulationName (CommandLineSimulationName);
        if (rc != SPARK_STATUS_SUCCESS) {
            fprintf (stderr, "main: RobotSpecifySimulationName() failed, rc=%d.\n", rc);
            exit (-3);
        }
    }

    //
    // Enter the main loop.
    //

    fprintf (stderr, "Press ^C to exit program--\n");

    FirstTimeStamp = RobotMillisecondTimeStamp ();

    while (TRUE) {
        RobotStartSimulation ();        // enable servers as detected.

        //
        // If the MISI channel isn't open, open it now.
        //

        if (MisiInputValueChannel == -1) {
            rc = RobotOpenMisiChannel ("Input Value", &MisiInputValueChannel);
            if (rc != SPARK_STATUS_SUCCESS) {
                fprintf (stderr, "main: RobotOpenMisiChannel('Input Value') failed, rc=%d.\n", rc);
                RobotDelayThreadMs (500);
            }
        }

        //
        // If the MOSI channel isn't open, open it now.
        //
                                                     
        if (MosiOutputValueChannel == -1) {
            rc = RobotOpenMosiChannel ("Output Value", &MosiOutputValueChannel);
            if (rc != SPARK_STATUS_SUCCESS) {
                fprintf (stderr, "main: RobotOpenMosiChannel('Output Value') failed, rc=%d.\n", rc);
                RobotDelayThreadMs (500);
                continue;
            }

            rc = RobotRegisterMosiActivationRtn (
                     MosiOutputValueChannel,
                     MosiActivationRtn,
                     MOSI_CONTEXT_OUTPUT_VALUE);

            if (rc != SPARK_STATUS_SUCCESS) {
                fprintf (stderr, "main: RobotRegisterMosiActivationRtn() failed, rc=%d.\n", rc);
                RobotDelayThreadMs (500);
                continue;
            }
        }

        //
        // Both channels are open. Stimulate the MISI channel.
        //
        
        StimulateMisiChannel ();
    }
    
    fprintf (stderr, "main: Exiting.\n");
    return 0;
} // main

//
// Private functions.
//

//
// FUNCTION NAME.
//      MosiActivationRtn - Receive MOSI Event From Simulation.
//
// FUNCTIONAL DESCRIPTION.
//      This function is called by the Robot Library when a registered
//      MOSI event occurs in the simulator, enabling the robot to perform
//      whatever real-world actions are required.
//
// MODIFICATION HISTORY.
//      S. E. Jones     20/09/16.       #1.0, new.
//      S. E. Jones     20/12/23.       #1.0, added dimension support.
//
// ENTRY PARAMETERS.
//      Context         - 64-bit context supplied in RobotRegisterMosiActivationRtn call.
//      NumberOfDimensions - dimensionality of MOSI channel; 0=single node, 1=linear array, 2=matrix, 3=volume.
//      Coordinates     - ptr to UINT16 array of relative offsets within channel to activated node.
//      Psp             - post-synaptic potential of node associated with MOSI channel.
//
//  EXIT PARAMETERS.
//      None.
//

static void MosiActivationRtn (
    IN UINT64 Context, 
    IN int NumberOfDimensions, 
    IN PUINT16 Coordinates, 
    IN int Psp
    )
{
    fprintf (
        stderr, 
        "MosiActivationRtn: Context=%"PRIu64", Dim=%u, Coordinates=(%u,%u,%u), Psp=%u.\n", 
        Context, NumberOfDimensions, Coordinates [0], Coordinates [1], Coordinates [2], Psp);
} // MosiActivationRtn

//
// FUNCTION NAME.
//      StimulateMisiChannel - Get Control to Stimulate MISI Channel.
//
// FUNCTIONAL DESCRIPTION.
//      This function is called by the main thread to periodically receive 
//      control to stimulate the simulation's MISI channel.
//
//      Because this routine can get called very often, here we get the
//      current timestamp and disallow sending MISI data more than once
//      per millisecond. In fact, this is a rate-based signal emitter,
//      so the signal we are monitoring is a number which modulates how
//      fast our pulse train's pulses are sent. Lower numbers cause more
//      time between pulses, while higher numbers cause less elapsed time
//      between pulses.
//
// MODIFICATION HISTORY.
//      S. E. Jones     21/01/11.       #1.0, new.
//
// ENTRY PARAMETERS.
//      None.
//
//  EXIT PARAMETERS.
//      None.
//

static void StimulateMisiChannel ()
{
    SPARK_STATUS rc;
    UINT64 TimeStamp;
    UINT64 Delta;

    //
    // If we don't have a MISI channel yet, don't stimulate it.
    //
    
    if (MisiInputValueChannel == -1) {
        return;
    }

    //
    // Create a sin(t) waveform, based on the current timestamp's distance to 
    // FirstTimeStamp.
    //
    
    TimeStamp = RobotMillisecondTimeStamp ();
    Delta = TimeStamp - FirstTimeStamp;
    MisiValue = sinf ((float)Delta / 360.0);
    
    if (0) fprintf (
        stderr, 
        "StimulateMisiChannel: TimeStamp = %" PRIu64 ", diff=%" PRIu64 ", MisiValue=%f.\n", 
        TimeStamp,
        Delta,
        MisiValue);

    //
    // If it's not time to emit a pulse, just return.
    //

    if (TimeStamp < NextMisiTimeStamp) {
        return;
    }
    
    //
    // We are ready to emit a pulse.
    //
    
    rc = RobotStimulateMisiChannel (MisiInputValueChannel);
    if (rc != SPARK_STATUS_SUCCESS) {
        fprintf (stderr, "StimulateMisiChannel: RobotStimulateMisiChannel() failed, rc=%d.\n", rc);
        return;
    }
    
    //
    // Schedule our next pulse.
    //
    
    fprintf (stderr, "StimulateMisiChannel: MisiValue = %f.\n", MisiValue);

    NextMisiTimeStamp = TimeStamp + (int)fabs (MisiValue * 100);

    // fprintf (stderr, "MISI TimeStamp Delta = %" PRIu64 ".\n", NextMisiTimeStamp - TimeStamp);
} // StimulateMisiChannel

//
// FUNCTION NAME.
//      Help - Display Help For Command Line Parameters.
//
// FUNCTIONAL DESCRIPTION.
//      This helper function displays the help for the command line parameters.
//
// MODIFICATION HISTORY.
//      S. E. Jones     20/09/26.       #1.0, new.
//      S. E. Jones     20/09/28.       #1.0, added network param.
//
// ENTRY PARAMETERS.
//      None.
//
//  EXIT PARAMETERS.
//      None.
//

static void Help ()
{
    fprintf (
        stderr,
        "Usage: ./Hello [-s simulation-name] [-u username] [-p password] [-h host] [--help]\n\n"
        "    -s simulation-name         Selects simulation to associate with.\n"
        "    -u username                Specifies username used to log-in to servers with.\n"
        "    -p password                Specifies password to be used with username.\n"
        "    -h host                    Specifies hostname or IP address of Sentience Server simulator.\n"
        "    --help                     Display this help text.\n"
        );
} // Help

This source code may be compiled with the following Linux command, producing an ELF binary Hello, which may be run from the Linux command line or as a headless Linux service.

gcc -g -pthread -o Hello Hello.c -lrobot -lrt -lm