1. Home
  2. Docs
  3. Getting started with MATR...
  4. Customizing MATRX
  5. Using a custom frontend

Using a custom frontend

Although the default MATRX visualizer can support quite a number of different HAT tasks with its top-down 2D grid view, there are even more that it doesn’t support. For instance, you could have:

Collaborative Intelligence Tool

A HAT task with an information agent where you don’t even need a 2D gridworld visualization, but instead want a dashboard, such as in this Collaborative Intelligence Tool.

An interface with a combination of a dashboard and the default 2D gridworld. Such as in this Meaningful Human Control (MHC) Triage tool (the code is open-source!). In this HAT task a topdown view is shown on the right using the default MATRX view of a hospital ward, and a dashboard with more in-depth patient information on the left.

Or just make some small additions, such as customizing the looks of the chat with colours or adding chat buttons. (Picture to be updated with chat view..)

All these examples show projects from TNO where MATRX was used with a custom or customized frontend. This tutorial will show how to achieve exactly that, by taking the default MATRX frontend and customizing it. As a running example we will use the MHC triage tool of which the code is open-source available.

Note: it is also possible to make an entirely different frontend, such as a frontend made with Unity that connects to the MATRX API. This will not be covered in this tutorial. But if you are interested, we suggest checking out the API docs here and reaching out to use if you require some help. MATRX has been connected to Unity before, so it is possible!

Customizing the default MATRX frontend

Customizing the default MATRX frontend enables you to make small changes, such as adding buttons to the chat, change colours in the styling, etc. The default MATRX frontend is a simple flask webserver that serves a Javascript frontend with bootstrap for all buttons and menu’s etc. The Javascript is connected to MATRX using the MATRX REST API. All of that means: if you want to edit the MATRX frontend, it is a plus if you know some HTML, CSS and/or Javascript. However, before we can edit the frontend, we’ll first have to copy the default MATRX frontend locally like so:

  • Copy the entire matrx_visualizer folder from the MATRX master branch on Git to your MATRX case root folder.
  • Rename the folder to something of your liking (e.g. my_customized_gui). In the MHC triage case, we renamed the matrx_visualizer folder to triage_gui.
  • In your WorldBuilder, disable the default MATRX visualizer by setting run_matrx_visualizer=False. For example how this is done in the MHC-triage case:
builder = WorldBuilder(shape=world_size, run_matrx_api=True,
                       run_matrx_visualizer=False,
                       visualization_bg_clr=bg_color,
                       visualization_bg_img="", tick_duration=config['world']['tick_duration'],
                       simulation_goal=AllPatientsTriaged(config['patients']['max_patients']))

  • Add some lines of code for starting and stopping your frontend as shown in the highlighted lines below. This is almost directly taken from the MHC-triage case here.
import os, requests

from my_customized_gui import visualization_server

if __name__ == "__main__":
    builder = your_case.create_builder()
    factory = WorldBuilder(random_seed=1, shape=[10, 10], tick_duration=config['world']['tick_duration'], verbose=False,
                           run_matrx_api=True, run_matrx_visualizer=False, visualization_bg_clr="#f0f0f0",
                           simulation_goal=100000000)

    # setup the media folder
    media_folder = os.path.abspath(os.path.realpath("your_media_folder"))
    builder.startup(media_folder=media_folder)

    # start the custom visualizer
    print("Starting custom visualizer")
    vis_thread = visualization_server.run_matrx_visualizer(verbose=False, media_folder=media_folder)

    # run the world
    world = builder.get_world()
    world.run(builder.api_info)

    # stop the custom visualizer
    print("Shutting down custom visualizer")
    r = requests.get("http://localhost:" + str(visualization_server.port) + "/shutdown_visualizer")
    vis_thread.join()

    # stop MATRX scripts such as the api and visualizer (if used)
    builder.stop()

Congratulations, you now have your own custom frontend! If we now start MATRX as usual, your new frontend should be used and you should see the print statements in the terminal 😀 Now we can get started with changing stuff:

On the left you can see a filetree of the matrx_visualizer. It contains quite a few files, but fortunatly you want have to fiddle with most!

Going from top to bottom:

  • The static/css folder contains files for the styling of MATRX. It is recommended not to change the main CSS file GUI.css, but to create your own custom CSS file and include that in the template you want to change, as done here. This will make it slightly easier to upgrade your frontend when there is a MATRX update that changed the frontend.
  • The static/images folder you can leave as is, as you can set a custom media folder by default in MATRX (see here for an example).
  • The static/js folder contains most of the rendering logic of the frontend. The two main files of interest here are gen_grid.js that parses the MATRX state in the draw function every tick, and the toolbar.js file which handles all functionality for the toolbar, including the chat. It is recommended not to change the other files.
  • The templates files contain the html templates with the page structure for artificial agents view, the god view, human agents view, and the start screen. In here you can change the structure of the page, such as to include the start/pause/stop buttons on agent views as well, or to include buttons at the bottom of the chat window.

So with this overview, how would you go about customizing the MATRX frontend? Here are a number of examples:

How to add chat buttons

Instead of having a chatbox in which you can type messages, it is also possible to make buttons that send hardcoded messages instead, or do a combination of both. In this example, we’ll look into adding a button that sends “hi” to all agents.

First, add a button to the bottom of the chat window in the human-agent template (see here). E.g., we can add the button below after the “send” button.

<button type="button" id="hi_button" class="btn btn-primary" onclick="my_send_message_function('hi')">Say Hi to everyone</button>

Now if you press the button, it will try to execute the Javascript function my_send_message_function with param hi, however this function doesn’t exist yet. We can see what it should approximately look like by peeking at the default send_message function which sends the contents of the chatbox here. If we change it slightly, we get the function we need:

function my_send_message_function(message) {
    data = {"content":message, "sender": lv_agent_id, "receiver": null} // setting receiver to null sends the message to all agents
    send_matrx_api_post_message(matrx_send_message_url, data);
}

Now pressing the button sends a message with content ‘Hi’ to all MATRX agents! Instead, you could also send it to a single agent ID or a list of agent IDs, or send more data by making the “content” in the data an object like so:

data = {"content":{"message": message, "param1":"val1", "param2":"val2"}, "sender": lv_agent_id, "receiver": "Agent_5"}

How to change chat message styling

In toolbar.js, there is a function called add_message that adds new chat messages to the screen. In that function you might filter for specific messages or specific text in messages, and give them a different class or styling. For instance, highlighting the word “Warning” in chat messages could be achieved by adding the code below to this line :

mssg_content.replace("Warning", "<span style='background-color:red;'>Warning</span>");

How to add the start/pause/stop buttons to the (human)agent view

These buttons are already present for the (human)agent views, but set to invisible. Enabling them is very simple, just remove the display:none; from these lines in the template html files and after clearing your cache and restarting MATRX it should work!

Tip: You can also enable only the play button for the human-agent view, so human testsubjects can only start the experiment but not pause or stop it!

How to make a (partial) dashboard interface

Let’s say you want both a dashboard with some interactivity on the left side and the default MATRX 2D gridworld, similar to the MHC triage tool. How would you do this?

  • First, add a custom css file and add it to the template you want to change, such as done here.
  • Then, change the template html file to make space for your dashboard. In the MHC triage case, you can see we added a wrapper (here) in which we have a div for the dashboard part of the interface (<div id="patient-cards"></div>), and the MATRX gridview (<div class="grid-container" id="container">). The grid-container is automatically filled by the gen_grid.js file to contain our default MATRX visualization. The patient-cards div is going to be filled by us to create our dashboard.
  • We fill our dashboard by adding custom code to the draw function in gen_grid.js, as that function processes the state of every agent every tick. In the example of the MHC triage tool, we check for objects of type patient in the new state and save it (see here). Instead, you could also call a custom function that puts the information of that patient as a new item in your patient-cards div.

You now have (the start of) a dashboard interface! Now you can style everything by making changes in your custom css file that you made in step 1, add other html to the template, or add custom JS for more interactivity..

If you want to go even further and just completely remove the 2D grid visualization and create your own (dashboard) frontend, that is also possible. You can remove everything from gen_grid.js, as long as you have the functions function draw(state, world_settings, new_messages, accessible_chatrooms, new_tick), function process_mssgs_pageload(initial_mssgs, initial_chatrooms), and function initialize_grid().

The first function is called every tick with the new state of your god or (human)agent view. The second and third function are called on pageload. These functions are called by loop.js and are required, but what you decide to do in them is totally up to you!

Help, it doesn’t work!

If you successfully created your own frontend, but don’t see the changes you make in the CSS, HTML or JS, it is most likely caused by your cache. Press ctrl + F5 to clear your old cache of the page and it should work. If that did not do the trick, or you have other questions, you can reach us via Github, or send us a mail at info@matrx-software.com!

Leave a Reply

Your email address will not be published. Required fields are marked *