Ahoy to my fellow MRL'ers!

I had an interesting thought while letting my turkey settle in my stomach (U.S. Thanksgiving Holiday Today). Is it or could it be possible to allocate resources for a specific service inside MRL when doing Runtime.createAndStart(...)  It is supposed to be possible to do java -Xmx1024m -jar myrobotlab.jar but does this just give the MyRobotLab process those resources?  I have been digging into Runtime and createAndStart but I am not seeing anything that looks explicitly like what I am wanting to do?  

If it is not currently a feature, can someone point me in the direction of where I would want to add in some sort of resources parameters? I am not seeing anywhere in the MRL code where it does something like a java -jar to start a specific service... 

After getting Junior to read I thought it would probably make it faster if I could give TesseractOCR a specific amount resouces. 

Many Thanks!

Kyle

GroG

6 years 4 months ago

Hey Kyle,
I'm excited about your distributed mrl cluster.  I've always been interested in this aspect of computing, and how a collection of computers can make a "bigger whole".

So first a few definitions.

  • Host - is a computer with a operating system
  • Operating System - software platform running on a Host which allows Application/Processes to run
  • Application/Process - a computer program which uses resources of the host.  These resources are usually memory, cpu cycles, and file storage.  Ultimately, the process requests these resources from the operating system
  • Thread - a little worker, which executes application program code.  Super simple applications have a single thread, but like us, we typically need to do a gazillion things at once.  So multi-threaded applications can do many things at once.
  • JVM - Java Virtual Machine - an abstraction of a Host + Operating system which allows applications to be (written once, and run anywhere). 
  • Inter-Process Communication - A process is isolated by the operating system, but its possible to communicate between different processes. One way is using the network to serialize messages from one process to another. Potentially this can be a very complicated procedure.  The goal usually is to try to hide the complexity, allowing developers to concentrate on "more interesting" work.

An Mrl Service is a multi-threaded functional unit which is dedicated to do "something" of interest.  Many services can exist within an Mrl Application/Process.  The  Mrl Application/Process runs on top of the JVM.  

When you start Mrl the JVM asks the operating system for resources.  And the amount & details of the resources can be specified by command line parameters.

A tricky part, is when you start Mrl, you don't start the services your interested in, rather you start the "Agent" process.  It's job is to construct a valid command line and spawn the "real" Mrl instance your interested in.

The Agent will spawn a new Mrl instance with default parameters if none are supplied.  The Agent has a lot of other capabilities, such as spawning multiple instances of mrl, keeping track of them, shutting them down, and potentially upgrading them.  This will be utilized more in the release after manticore.

If you look at ProcessData class - you can see all the bits and parts the Agent controls.

One of which are command line parameters to the JVM

If your interested in creating an Mrl Cluster of hosts, I would recommend the following rule.

Take 1/2 your host's memory it to a single process MRL process running on the host.  Although you can run multiple processes on a single hosts, usually there is no performance benefit to this.  The services within the process will share the dedicated resources.  Splitting this up eats up a bit of the memory anyway, and the services are left with fragmented memory.

You can give 1/2 of the memory to the Mrl process with the following command if your system has 2Gigs of memory.

java -jar myrobotlab.jar -jvm "-Xmx1024m -Xms1024m -Djava.library.path=libraries/native -Djna.library.path=libraries/native -Dfile.encoding=UTF-8"

This tells the mrl instance to get 1Gig memory, and get it "now" - ie the jvm allocates it from the os at startup 

This is the result we can see in a windows process explorer (same info could be done by Linux "top" command)

In a clustered environment this is the simplest and typically the most performant setup

I have a lot of changes which are waiting post manticore release, many have to do with simplifying the connectivity between mrl processes.  In addition, there are more changes which will allow easier connectivity between mrl and non-java applications.  Your success with Mqtt is great, and I'm excited to see what further things you will connect with.  
Mrl's internal mechanism of pub/sub has some similarities to mqtt and I hope to meld these concepts together in the next release.  Additionally, I intend to add Moquette mqtt broker as a service in Mrl.