Skip to content

Latest commit

 

History

History
694 lines (553 loc) · 32.9 KB

README.md

File metadata and controls

694 lines (553 loc) · 32.9 KB

Rainbow

Rainbow CSharp SDK v3 - Bot Library and Bot examples

You will find here several samples which illustrate how to use the Rainbow CSharp SDK to create Bot.

They are all working in Linux, MacOs or Windows.

They are listed in order of priority / difficulty if you just started to use the SDK and/or create Bot.

All bots are using a state machine to simplify the complexity and inherits from Bot Base

Table of content

Prerequisites

Rainbow API HUB

This SDK is using the Rainbow environment

This environment is based on the Rainbow service

Rainbow CSharp SDK

To have more info about the SDK:

State Machine

A third party library (stateless is used to create the state machine.

Several library exists. This one has been choosed because it's a lightweight one (no dependency) and permits to visualize the state machine using a dot graph like the one below.

The dot graph can be direclty created from the state machine defined by code using only one line:

StateMachine<State, Trigger> _machine; // The state machine
...
String dotGrpah = UmlDotGraph.Format(_machine.GetInfo()); // Create dot graph as String once the state machine has been totally defined

A dot graph is a plain text describing a graph. To visualize it a tool is necessary like these ones:

Bot Base

This bot is used as base for all other bots (using inheritance)

It's using three JSON configuration files:

  • exeSettings.json: to define mandatory settings for the application - data structure same for all bots
  • credentials.json: to define mandatory settings to connect to Rainbow server - data structure same for all bots
  • botConfiguration.json: to define settings used by the bot - data structure can be enhanced according bot features

It provides a BotLibrary.BotBase class which must be inherited according your Bot features. This class provides default features (connection, reconnection, ...) to avoid to implement them each time in you bots.

File exeSettings.json

This file must be stored in ./config folder path. This online tool can be used to ensure that a valid JSON is used.

The structure of this file is always the same. It looks like this:

{
  "exeSettings": {
    "useAudioVideo": false,
    "ffmpegLibFolderPath": "C:\\ffmpeg-7.0.1-full_build-shared\\bin",
    "logFolderPath": ".\\logs"
  }
}

Details:

  • useAudioVideo: Boolean (false by default) - True if the current bot is using audio / video medias.
  • ffmpegLibFolderPath: String (null/empty by default) - Must be set to a valid folder path to FFmpeg libraries if useAudioVideo is set to true.
  • logFolderPath: String (".\logs" by default) - Folder path where logs of the bots will be stored.

File credentials.json

This file must be stored in ./config folder path. This online tool can be used to ensure that a valid JSON is used.

The structure of this file is always the same. It looks like this:

{
  "credentials": {
    "serverConfig": {
      "appId": "TO DEFINE",
      "appSecret": "TO DEFINE",
      "hostname": "TO DEFINE"
    },

    "userConfig": {
      "iniFolderPath": ".\\logs",
      "prefix": "BOT1",
      "login": "my_account@my_domain.com",
      "password": "my_password"
    }
  }
}

Details:

  • serverConfig: See developer journey for more details
    • appId: Application Id (its value is dependent of the host name used)
    • appSecret: Application Secret(its value is dependent of the host name used)
    • hostname: Host name used to connect to the Rainbow server (examples: openrainbow.com, openrainbow.net, ...)
  • userConfig:
    • iniFolderPath: String (".\logs" by default) - Folder path where INI file of the bot will be stored.
    • prefix: String (must no be empty/null) - Prefix used for the bot for logging purpose
    • login: Login (email address) used by the bot to connect to Rainbow server
    • password: Password used by the bot to connect to Rainbow server

File botConfiguration.json

This file must be stored in ./config folder path. This online tool can be used to ensure that a valid JSON is used.

The data structure of this file can be enhanced according bot features. Here the minimal/mandatory structure is described.

{
  "botConfiguration": {
    "administrators": {
      "rainbowAccounts": [
        {
          "id": "123456454",
          "jid": "[email protected]",
          "login": "my_user1@my_domain.com"
        },
        {
          "id": "455678123546",
          "jid": "[email protected]",
          "login": "my_user2@my_domain.com"
        }
      ],
      "guestsAccepted": false
    },
    "bubbleInvitationAutoAccept": true,
    "userInvitationAutoAccept": true
  }
}

Details:

  • administrators:
    • rainbowAccounts: To define one or several Rainbow accounts which are allowed to send commands to the bot (in this example two are defined). If received by someone else the command is not taken into account.
      • id: String (can be null/empty or not set) - Id of the Rainbow Account.
      • jid: String (can be null/empty or not set) - Jid of the Rainbow Account.
      • login: String (can be null/empty or not set) - Login of the Rainbow Account.
      • NOTE: id is used first then jid and finally login. At least one of them must be set.
    • guestsAccepted: Boolean (false by default) - If set to True any guest account can send command to the bot.
  • bubbleInvitationAutoAccept: Boolean (true by default) - If set to true, the bot will accept automatically of Bubble invitations.
  • userInvitationAutoAccept: Boolean (true by default) - If set to true, the bot will accept automatically of User invitations.
  • bot: To define which bot must take into account this configuration (can be null/empty or not set - so will target any bot)
    • id: String (can be null/empty or not set) - Id of the Rainbow Account used by the bot.
    • jid: String (can be null/empty or not set) - Jid of the Rainbow Account used by the bot.
    • login: String (can be null/empty or not set) - Login of the Rainbow Account used by the bot.

BotLibrary.BotBase class

BotLibrary.BotBase class implements this state machine: Rainbow

It's dot graph is available here

Implemented features

Thanks to the state machine, features available are:

All these features are available to any Bot inheriting from BotLibrary.BotBase class

Methods to override

BotLibrary.BotBase class provides several methods which can be overrided (it's not mandatory) according Bot features.

Methods available which can be overridden are:

// Called when the Bot is connected to Rainbow server (called also after reconnection)
public virtual async Task ConnectedAsync()

// Called when the Bot has been stopped (after too many auto-reconnection attempts or after a logout)
public virtual async Task StoppedAsync(SdkError sdkerror)

// Called when a Bubble Invitation is received if bubbleInvitationAutoAccept setting is not set to true
public virtual async Task BubbleInvitationReceivedAsync(BubbleInvitation bubbleInvitation)

// Called when a Bubble Invitation is received if userInvitationAutoAccept setting is not set to true
public virtual async Task UserInvitationReceivedAsync(Invitation invitation)

// Called when an AckMessage is received. If it contains a BotConfiguration update, BotConfigurationUpdatedAsync method is called instead
public virtual async Task AckMessageReceivedAsync(AckMessage ackMessage)

// Called when an ApplicationMessage is received. If it contains a BotConfiguration update, BotConfigurationUpdatedAsync method is called instead
public virtual async Task ApplicationMessageReceivedAsync(ApplicationMessage applicationMessage)

// Called when an InstantMessage is received. If it contains a BotConfiguration update, BotConfigurationUpdatedAsync method is called instead
public virtual async Task InstantMessageReceivedAsync(Message message)

// Called when an InternalMessage is received. If it contains a BotConfiguration update, BotConfigurationUpdatedAsync method is called instead
public virtual async Task InternalMessageReceivedAsync(InternalMessage internalMessage)

// Called when a BotConfiguration update has been received
public virtual async Task BotConfigurationUpdatedAsync(BotConfigurationUpdate botConfigurationUpdate)

To override a method just do something like this: (here an example using only StoppedAsync() and InstantMessageReceivedAsync() methods)

// We use a specific namespace (MyNamespace) which contains your Bot (named MyBot)
namespace MyNamespace
{
    // MyBot inherits from BotLibrary.BotBase
    public class MyBot: BotLibrary.BotBase
    {
        // We override the method StoppedAsync() - notice 'override' keyword and also the use of 'async Task'
        public override async Task StoppedAsync(SdkError sdkerror)
        {
            // Add here your own logic when your bot is stopped

            // If don't use any async method here you can use this code to avoid warning
            // await Task.CompletedTask;
        }

        // We override the method InstantMessageReceivedAsync() - notice 'override' keyword and alsot the use of 'async Task'
        public override async Task InstantMessageReceivedAsync(Message message)
        {
            // Add here your own logic when an IM is received

            // If don't use any async method here you can use this code to avoid warning
            // await Task.CompletedTask;
        }
    }
}

For concrete examples, check code of Bot BasicMessages and Bot Broadcaster to see how they override methods to provide their own features.

Public Properties / Methods available

There is also several public properties / methods available in BotLibrary.BotBase class.

/// To provide access to the Rainbow.Application object used by the bot. Permit to use all SDK C# features from it
public Rainbow.Application Application;

/// To get the name of the Bot (it's the prefix defined in file credentials.json file)
public String BotName;

/// To configure the bot - must be called before to use Login()
public async Task<Boolean> Configure(JSONNode jsonNodeCredentials, JSONNode jsonNodeBotConfiguration);

/// Once configured, to start login process
public Boolean Login();

/// To start logout process - StoppedAsync() will be called once done
public void Logout();

/// To add an 'InternalMessage' to the queue. It will be dequeued using InternalMessageReceivedAsync(InternalMessage)'
/// It permits by code to send a BotConfiguration Update or any other data
public void AddInternalMessage(InternalMessage internalMessage);

/// To know if Contact specified is an administrator of this bot
public Boolean IsAdministrator(Contact? contact);

/// To know if Jid specified is administrator of this bot. if this jid is not in the cache, ask the server more info
public async Task<Boolean> IsAdministrator(String? jid);

/// To know if the bot is stopped and why
public (Boolean isStopped, SdkError? sdkError) IsStopped();

/// Get Dot Graph of this Bot
public String ToDotGraph();

/// Get the current state of the Bot (i.e. state of the state machine)
public State GetState();

/// Get the current state of the Bot (i.e. state of the state machine) and the trigger used to reach it
public (State, Trigger) GetStateAndTrigger();

/// Determine if the Bot is in the supplied state
public Boolean IsInState(State state);

/// To check / get the bot status
public (BotLibrary.State state, Boolean canContinue, String message) CheckBotStatus();

BotConfiguration update

The term BotConfiguration update means that the configuration of the Bot has been updated (by AckMessage, ApplicationMessage, InstantMessage or InternalMessage) or at startup (using Configure() method).

This update is based on a JSON structure defined in file botConfiguration.json.

The method BotConfigurationUpdatedAsync() is called with BotConfigurationUpdate object as parameter.

This object is defined like this:

    public class BotConfigurationUpdate
    {
        /// Unique identifier
        public String Id { get; set; }

        /// JSONNode describing the bot configuration
        public JSONNode JSONNodeBotConfiguration { get; set; }

        /// Describe the update context. 
        /// Can be "internalMessage", "ackMessage", "instantMessage", "applicationMessage" or "configFile"
        /// "configFile" is used when the bot is finally connected and permits to have the configuration set as file at startup. There is no ContextData is this case.
        public String Context {  get; set; }

        /// Provide the context data as object - to know it's type use Context property
        /// Can be an AckMessage object, an ApplicationMessage object, a Message object (in InstantMessage context) or InternalMessage object or null
        public Object? ContextData { get; set; }
    }
}

So using Context and ContextData properties, it's possible to know in which context the update is performed.

JSONNodeBotConfiguration property as JSONNode object contains all data provided in the update.

If the data provided contains only the JSON structure defined in file botConfiguration.json, the BotLibrary.BotBase class will update the configuration by itself. So it's possible to update administrators list, bubble/user invitations and any settings defined in the default structure. It's not necessary to provide all data structure. Only data specified will be updated.

If you provide more data that, you have to manage them using your own implementation. Bot Broadcaster is using this principle to offer new features (media stream selection, which conference must be joined). Check this bot for mode details

How to send a BotConfiguration update

Using AckMessage

AckMessage can be sent by code only.

Code example using SDK C#:

String json;        // a JSON with the same structure as defined in file botConfiguration.json
Peer peerContact;   // valid Peer as Contact
String resource;    // valid resource for this Peer

Rainbow.Application rbApplication; // A valid Rainbow.Application object (with an account already connected)

var instantMessaging = rbApplication.GetInstantMessaging();
await instantMessaging.SendAckMessageAsync(MessageType.Set, peerContact, resource, "botConfiguration", HttpRequestDescriptor.MIME_TYPE_JSON, json);

Using ApplicationMessage

ApplicationMessage can be sent by code only.

Code example using SDK C#:

String json;// a JSON with the same structure as defined in file botConfiguration.json
Peer peer;  // valid Peer as Contact or as Bubble

Rainbow.Application rbApplication; // A valid Rainbow.Application object (with an account already connected)

System.Xml.XmlElement botConfiguration = new XmlDocument().CreateElement("botConfiguration");
botConfiguration.InnerText = Rainbow.Util.StringWithCDATA(json);

var instantMessaging = rbApplication.GetInstantMessaging();
await instantMessaging.SendApplicationMessageAsync(peer, [botConfiguration]);

Using InstantMessaging

InstantMessaging can be sent by code but you can also use any Rainbow CPaaS application.

If want to use a Rainbow CPaaS application, you just have to send an IM (in P2P to the bot or in a bubble where the bot is a member) with a file with ".json" as extension and with the structure defined in File botConfiguration.json

Code example using SDK C# and a file:

String filePath;    // a path to a file with ".json" as extension and with JSON using the same structure as defined in file botConfiguration.json
Peer peer;          // valid Peer as Contact or as Bubble

Rainbow.Application rbApplication; // A valid Rainbow.Application object (with an account already connected)

var instantMessaging = rbApplication.GetInstantMessaging();
await instantMessaging.SendMessageWithFileAsync(peer, null, filePath);

Code example using SDK C# and a String:

String json;// a JSON with the same structure as defined in file botConfiguration.json
Peer peer;  // valid Peer as Contact or as Bubble

Rainbow.Application rbApplication; // A valid Rainbow.Application object (with an account already connected)

var instantMessaging = rbApplication.GetInstantMessaging();
MemoryStream memoryStream = new(Encoding.UTF8.GetBytes(json));
await instantMessaging.SendMessageWithStreamAsync(peer, null, memoryStream, "botConfiguration.json");

Using InternalMessage

InternalMessage can be sent by code only.

Code example using SDK C#:

String json;        // a JSON with the same structure as defined in file botConfiguration.json
Peer peer;          // valid Peer as Contact or as Bubble

MyBot mybot;        // A valid mybot object which inherits from BotLibrary.BotBase class and which is connected

InternalMessage internalMessage = new()
    {
        Type = "botConfiguration",
        Data = json
    };
await mybot.AddInternalMessage(internalMessage);

Bot BasicMessages

This bot demonstrates how to inherit from BotLibrary.BotBase class and overrides three methods.

It permits to create a bot in ~50 lines which implements all features of Bot Base and also to answer to all AckMessage, ApplicationMessage and InstantMessage.

The full code is:

using Rainbow;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Xml;

namespace BotBasic
{
    public class BotBasicMessages: BotLibrary.BotBase
    {
#region Messages - AckMessage, ApplicationMessage, InstantMessage, InternalMessage
        public override async Task AckMessageReceivedAsync(Rainbow.Model.AckMessage ackMessage)
        {
            // Here we answer to all AckMessage using a default Result message
            await Application.GetInstantMessaging().AnswerToAckMessageAsync(ackMessage, Rainbow.Enums.MessageType.Result);
        }

        public override async Task ApplicationMessageReceivedAsync(Rainbow.Model.ApplicationMessage applicationMessage)
        {
            // Here we answer to all ApplicationMessage using a default message

            String senderDisplayName = await GetSenderDisplayName(applicationMessage.FromJid);

            // Create and send an ApplicationMessage as answer
            List<XmlElement> xmlElements = [];
            var el1 = new XmlDocument().CreateElement("elm1");
            el1.InnerText = Rainbow.Util.StringWithCDATA($"Hi, It's an auto-answer from 'BotBasic' SDK C# example. ApplicationMessage has been sent by [{senderDisplayName}].");
            xmlElements.Add(el1);
            await Application.GetInstantMessaging().AnswerToApplicationMessageAsync(applicationMessage, xmlElements);
        }

        public override async Task InstantMessageReceivedAsync(Rainbow.Model.Message message)
        {
            // Here we answer to InstantMessage using a default message

            String senderDisplayName = await GetSenderDisplayName(message.FromContact?.Peer?.Jid);

            // Create and send an answer
            String answer = $"Hi, It's an auto-answer from 'BotBasic' SDK C# example. InstantMessage received has been sent by {senderDisplayName}.";
            await Application.GetInstantMessaging().AnswerToMessageAsync(message, answer);
        }
    }
#endregion Messages - AckMessage, ApplicationMessage, InstantMessage, InternalMessage

        private async Task<String> GetSenderDisplayName(String? jid)
        {
            // Get the sender as Contact
            var contact = await Application.GetContacts().GetContactByJidInCacheFirstAsync(jid);
            if (String.IsNullOrEmpty(contact?.Peer?.DisplayName))
                return "an unknown contact";
            else
                return $"[{contact.Peer.DisplayName}]";
        }
}

Bot Broadcaster

Thanks to inheritance, this bot provides all features of Bot Base.

It permits also to broadcast medias in a specific conference so it's necessary to specified useAudioVideo as true and a valid value for ffmpegLibFolderPath as detailed in file exeSettings.json.

Extended BotConfiguration

To specify which stream/media is broadcasted and in which conference, the structure of file botConfiguration.json as been extended.

{
  "botConfiguration": {
    "administrators": {
      "rainbowAccounts": [
        {
          "id": "123456454",
          "jid": "[email protected]",
          "login": "my_user1@my_domain.com"
        }
      ],
      "guestsAccepted": false
    },
    "bubbleInvitationAutoAccept": true,
    "userInvitationAutoAccept": true,

    "streams": [
      {
        "id": "id1",
        "media": "audio",
        "uri": "c:\\media\\myAudioFile.mp4",
        "connected": false
      },
      {
        "id": "id2",
        "media": "video",
        "uri": "c:\\media\\myVideoFile.mp4",
        "connected": false
      },
      {
        "id": "id3",
        "media": "video",
        "uri": "rtsp://rtsp.stream/movie",
        "uriSettings": {
          "rtsp_transport": "tcp",
          "max_delay": "0"
        },
        "forceLiveStream": false,
        "connected": false
      },
      {
        "id" : "screen1",
        "media" : "video",
        "uri" : "Composite GDI Display",
        "uriType" : "screen"
      },
      {
        "id" : "webcam1",
        "media" : "video",
        "uri" : "HD Pro Webcam C920",
        "uriType" : "webcam"
      },
      {
        "id" : "microphone1",
        "media" : "audio",
        "uri" : "Microphone (HD Pro Webcam C920)",
        "uriType" : "microphone"
      }
      ,
      {
        "id" : "composition1",
        "media" : "composition",
        "videoComposition" : [ "id2", "webcam1"],
        "videoFilter" : "[1]setpts=PTS-STARTPTS,scale=640:-1[scaled0];[0]setpts=PTS-STARTPTS,scale=250:-1[scaled1];[scaled0][scaled1]overlay=main_w-overlay_w-10:main_h-overlay_h-10"
      }
    ],

    "conference": {
      "id": "632469c100e14c9bf133e889",
      "jid": "632469c100e14c9bf133e889",
      "name": "CCTV Bubble",
      "audioStreamId": "id1",
      "videoStreamId": "id2",
      "sharingStreamId": "id2"
    }
  }
}

Details:

  • administrators object, bubbleInvitationAutoAccept and userInvitationAutoAccept: see file botConfiguration.json
  • streams: to define one of several streams and how to connect / use them
    • id: String (cannot be null/empty) - Unique identifier of the stream
    • media: String (possible values: "audio", "video", "audio+video", "composition")
      • Media to grab from this stream
      • If the stream contains audio and video but you want only to use it for video purpose use "video". It permits to avoid to decode audio part.
      • "composition" permits to create a video stream using one of several streams (see videoComposition) based on a videoFilter
    • uriType: String (possible values: "other" (default value if null or not provided), "screen", "webcam", "microphone")
      • to define the type of the uri
      • "screen" to grab stream from a plugged screen, "webcam" to grab stream from a plugged webcam, "microphone" to grab stream from a plugged microphone.
      • "other" (or null or not provided) to grab stream from any other source.
    • uri: String (cannot be null/empty) - Uri to use to connect to the stream
      • can be a path to a local or remote file - for example "c:\media\myVideoFile.mp4"
      • can be an URL (http, ftp, rtsp, ...). You can specify a login/pwd like this: rtsp://myLogin:[email protected]/movie
      • can be the full name of the local device to use (when "screen", "webcam" or "microphone" are used for uriType)
      • can be null or not provided if a "composition" is used
    • uriSettings: Object (can be null or not provided)
      • More settings to enhance connection to the stream.
      • One or several settings can be set. It's related to the protocol used and options available in ffmpeg.
    • connected: Boolean (can be null or not provided - false by default) - If True it means the bot will connect and stay connected to the stream even if this stream is not used in the conference or if there is no conference.
    • forceLiveStream: Boolean (can be null or not provided - null by default) - Due to some internal restrictions, remote stream could be not grabbed well. Using true or false can help to resolve this situation.
    • videoComposition: Array of String (can be null or not provided - null by default)
      • Mandatory if media is set as "composition"
      • The array contains id of video stream used to create the composition. Order used is important and it's related to the videoFilter
    • videoFilter: String (can be null or not provided - null by default)
      • Mandatory if media is set as "composition"
      • Can be used also if media is set as "video", "audio+video" to use a filter of the original stream
      • If used a valid filter for ffmpeg must be used - see ffmpeg video filters documentation
  • conference:
    • To define the conference to use. If null or not provided, the bot will not be a conference.
    • if a conference is defined but it's not yet active, the bot will wait and join it when it's started.
    • if a conference is defined but cannot be found by the bot, it will not join the conference.
    • id: String (cannot be null/empty) - Id of the bubble/conference
    • jid: String (cannot be null/empty) - Jid of the bubble/conference
    • name: String (cannot be null/empty) - Full name of the bubble/conference
    • NOTE: At east one of id, jid or name must be defined.
    • audioStreamId: String (cannot be null/empty) - Id of the stream (defined in streams) to use as Audio stream
    • videoStreamId: String (cannot be null/empty) - Id of the stream (defined in streams) to use as Video stream
    • sharingStreamId: String (cannot be null/empty) - Id of the stream (defined in streams) to use as Sharing stream
    • NOTE: if audioStreamId, videoStreamId are sharingStreamId are not defined or all badly set, the bot will not join the conference since there is no valid stream.

Features

Using this extended data structure and sending a BotConfiguration update, it's now possible:

  • to add / remove a Bot in a conference
  • to add / remove streams used by the Bot in the conferenc: Audio, Video, Sharing, Audio+Video, Audio+Sharing, Video+Sharing, Audio+Video+Sharing
  • to create a video composition for one or several video streams

BotConfiguration update using extended data

As explained in the chapter BotConfiguration update, the method BotConfigurationUpdatedAsync() is called with BotConfigurationUpdate object as parameter.

So the Bot Broadcaster overrides this method and implements it's own logic:

public override async Task BotConfigurationUpdatedAsync(BotConfigurationUpdate botConfigurationUpdate)
{
    // Ensure to deal with an object not null
    if (botConfigurationUpdate is null)
        return;

    // BotConfigurationExtended object has been created to store data structure specific for this bot
    // We try to parse JSON Node to fill this data structure and if it's correct we update the broadcast configuration
    if (BotConfigurationExtended.FromJsonNode(botConfigurationUpdate.JSONNodeBotConfiguration, out BotConfigurationExtended botConfigurationExtended))
    {
        lock (_lockBotConfiguration)
            _nextBotConfigurationUpdate = botConfigurationUpdate;

        UpdateBroadcastConfiguration();
    }

    await Task.CompletedTask;
}

Details:

  • botConfigurationUpdate.JSONNodeBotConfiguration contains all the data sent as BotConfiguration update using Extended BotConfiguration.
  • BotConfigurationExtended object using FromJsonNode() method is used to validate the data.
  • If it's correct the new configuration is stored and the broadcast/conference settings is updated in consequence.

To have more details: check Bot Broadcaster code, README and CHANGELOG