TheLazy.Dev
Portfolio GitHub

Blog

lumberyard |gamedev |rust |ffi

Lumberyard EBus from Rust

jeikabu

jeikabu

Posted on December 6, 2019

Lumberyard EBus from Rust

One of the architectural changes Amazon made to CryEngine is the introduction of the EBus (Event Bus):

a general-purpose communication system that Lumberyard uses to dispatch notifications and receive requests

Ebuses are a key participant in the component and gem systems and used throughout the engine. Their indirection and de-coupling make them an ideal candidate for integrating additional languages in Lumberyard development.

For further details about ebus, see:

EBus Primer

Let’s take a look at using the TickBus:

#include <AzCore/Component/TickBus.h>

class MyEbusHandlerComponent
    : public Component
    , public AZ::TickBus::Handler
{

private:
    void Activate() override
    {
        // Connect to the bus
        TickBus::Handler::BusConnect();
    }

    void OnTick(float deltaTime, ScriptTimePoint time) override
    {
        // Handle tick event
    }
};

// Send a message on `TickRequestBus`
float frameDeltaTime = 0.f;
TickRequestBus::BroadcastResult(frameDeltaTime, &AZ::TickRequestBus::Events::GetTickDeltaTime);
Enter fullscreen mode Exit fullscreen mode

To subscribe to an ebus and receive published events:

  1. Derive from EBus<T>::Handler
  2. Call EBus<T>::Handler::BusConnect()

Broadcast() and BroadcastResult() are used to send messages (requests) to an ebus.

Down the Template Rabbit Hole

Let’s take a look at what’s going on by unraveling the innocuous looking:

TickBus::Handler::BusConnect();
Enter fullscreen mode Exit fullscreen mode

TickBus is defined in Code/Framework/AzCore/AzCore/Component/TickBus.h:

/**
* The EBus for tick notification events.
* The events are defined in the AZ::TickEvents class.
*/
typedef AZ::EBus<TickEvents> TickBus;
Enter fullscreen mode Exit fullscreen mode

Ok, so we’re working with an EBus. EBus is defined in Code/Framework/AzCore/AzCore/EBus/EBus.h:

template<class Interface, class BusTraits = Interface>
class EBus
    : public BusInternal::EBusImpl<AZ::EBus<Interface, BusTraits>, BusInternal::EBusImplTraits<Interface, BusTraits>, typename BusTraits::BusIdType>
{
    //...
Enter fullscreen mode Exit fullscreen mode

Because only one template parameter is specified, both Interface and BusTraits template parameters are the same type (i.e. EBus<TickEvents, TickEvents>).

EBusImpl is defined in Code/Framework/AzCore/AzCore/EBus/BusImpl.h:

template <class Bus, class Traits>
struct EBusImpl<Bus, Traits, NullBusId>
    : public EventDispatcher<Bus, Traits>
    , public EBusBroadcaster<Bus, Traits>
    , public EBusBroadcastEnumerator<Bus, Traits>
    , public AZStd::Utils::if_c<Traits::EnableEventQueue, EBusBroadcastQueue<Bus, Traits>, EBusNullQueue>::type
{
};

//...

template <class Bus, class Traits>
struct EBusBroadcaster
{
    /**
    * An event handler that can be attached to only one address at a time.
    */
    using Handler = typename Traits::BusesContainer::Handler;
};
Enter fullscreen mode Exit fullscreen mode

It inherits from a few things, but the one we care about is EBusBroadcaster which defines Handler.

So, to summarize what we have figured out so far:

//TickBus::Handler::BusConnect();
EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();
Enter fullscreen mode Exit fullscreen mode

EBusImplTraits is defined in BusImpl.h

template <class Interface, class BusTraits>
struct EBusImplTraits
{
    //...

    /**
    * Contains all of the addresses on the EBus.
    */
    using BusesContainer = AZ::Internal::EBusContainer<Interface, Traits>;
Enter fullscreen mode Exit fullscreen mode

That means we now know:

//TickBus::Handler::BusConnect();
//EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();
AZ::Internal::EBusContainer<_, TickEvents>::Handler::BusConnect();
Enter fullscreen mode Exit fullscreen mode

Code/Framework/AzCore/AzCore/EBus/Internal/BusContainer.h:

// Default impl, used when there are multiple addresses and multiple handlers
template <typename Interface, typename Traits, EBusAddressPolicy addressPolicy = Traits::AddressPolicy, EBusHandlerPolicy handlerPolicy = Traits::HandlerPolicy>
struct EBusContainer
{
public:
    //...

    using Handler = IdHandler<Interface, Traits, ContainerType>;
Enter fullscreen mode Exit fullscreen mode

Thing is, IdHandler doesn’t define BusConnect(). The comment gives you a clue about what’s going on- specialization/partial specialization based on AddressPolicy and HandlerPolicy from the trait (i.e. TickRequests).

TickEvents in TickBus.h:

/**
* Interface for AZ::TickBus, which is the EBus that dispatches tick events.
* These tick events are executed on the main game thread. In games, AZ::TickBus
* dispatches ticks even if the application is not in focus. In tools, AZ::TickBus 
* can become inactive when the tool loses focus.
* @note Do not add a mutex to TickEvents. It is unnecessary and typically degrades performance.
*/
class TickEvents
    : public AZ::EBusTraits
{
    //...
Enter fullscreen mode Exit fullscreen mode

EBusTraits in EBus.h:

struct EBusTraits
{
public:
    /**
        * Defines how many handlers can connect to an address on the EBus
        * and the order in which handlers at each address receive events.
        * For available settings, see AZ::EBusHandlerPolicy.
        * By default, an EBus supports any number of handlers.
        */
    static const EBusHandlerPolicy HandlerPolicy = EBusHandlerPolicy::Multiple;

    /**
        * Defines how many addresses exist on the EBus.
        * For available settings, see AZ::EBusAddressPolicy.
        * By default, an EBus uses a single address.
        */
    static const EBusAddressPolicy AddressPolicy = EBusAddressPolicy::Single;
Enter fullscreen mode Exit fullscreen mode

Therefore the specialization we’re interested in is EBusContainer<_, _, EBusAddressPolicy::Single, EBusHandlerPolicy::Multiple>:

// Specialization for single address, multi handler
template <typename Interface, typename Traits, EBusHandlerPolicy handlerPolicy>
struct EBusContainer<Interface, Traits, EBusAddressPolicy::Single, handlerPolicy>
{
public:
    using ContainerType = EBusContainer;
    using IdType = typename Traits::BusIdType;

    //...

    using Handler = NonIdHandler<Interface, Traits, ContainerType>;
Enter fullscreen mode Exit fullscreen mode

Now we finally know what TickBus::Handler is:

//TickBus::Handler::BusConnect();
//EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();
//AZ::Internal::EBusContainer<_, TickEvents>::Handler::BusConnect();
NonIdHandler<_, TickEvents>::BusConnect();
Enter fullscreen mode Exit fullscreen mode

Look at definition of NonIdHandler in Code/Framework/AzCore/AzCore/EBus/Internal/Handlers.h:

/**
* Handler class used for buses with only a single address (no id type).
*/
template <typename Interface, typename Traits, typename ContainerType>
class NonIdHandler
    : public Interface
{
public:
    using BusType = AZ::EBus<Interface, Traits>;
    //...

    void BusConnect();
    void BusDisconnect();
Enter fullscreen mode Exit fullscreen mode

Sure enough, there’s the declaration for BusConnect(). But, what about the definition? That’s back in Ebus.h:

template <typename Interface, typename Traits, typename ContainerType>
void NonIdHandler<Interface, Traits, ContainerType>::BusConnect()
{
    typename BusType::Context& context = BusType::GetOrCreateContext();
    AZStd::scoped_lock<decltype(context.m_contextMutex)> contextLock(context.m_contextMutex);
    if (!BusIsConnected())
    {
        typename Traits::BusIdType id;
        m_node = this;
        BusType::ConnectInternal(context, m_node, id);
    }
}
Enter fullscreen mode Exit fullscreen mode

this is the AZ::TickBus::Handler-derived handler instance from which we called BusConnect() (e.g. in Activate()). This looks like the end, let’s just pinch this off: it grabs a lock and calls EBus<>::ConnectInternal() (also in Ebus.h), which calls Connect() on EBusContainer instance member (back in BusContainer.h), and adds our handler to the list of handlers.

All that template shenanigans to add a pointer to a list. Good times.

Rust-ified

All of that was very interesting, but how can we call it from Rust? Bindgen doesn’t support several “advanced” C++ techniques like template functions, partial template specialization, and traits templates.

This results in a few problems for us:

  1. No way to resolve types

    • As we found above, because of template specialization TickBus::Handler::BusConnect() is NonIdHandler<_, TickEvents>::BusConnect(). But Bindgen only generates bindings for the base template: 
    pub type EBusContainer_Handler<Interface> = root::AZ::Internal::IdHandler<Interface>;

  2. There’s no functions to bind (link) to

    • Almost everything is defined via templates in header files; mostly inlined and not externally visible symbols

The simplest approach is wrapping the needed routines in “plain” C functions like we did to deal with Lumberyard Editor IPlugin:

#include <AzCore/Component/TickBus.h>
#include <AzCore/Script/ScriptTimePoint.h>

extern "C" {
    void TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(AZ::ScriptTimePoint& results)
    {
        AZ::TickRequestBus::BroadcastResult(results, &AZ::TickRequestBus::Events::GetTimeAtCurrentTick);
    }
}
Enter fullscreen mode Exit fullscreen mode

In Rust:

extern "C" {
    pub fn TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(results: *mut root::AZ::ScriptTimePoint);
}
Enter fullscreen mode Exit fullscreen mode

Simple and effective, albeit tedious. But we’re not done yet.

Look at the definition of ScriptTimePoint in Code\Framework\AzCore\AzCore\Script\ScriptTimePoint.h:

class ScriptTimePoint
{
public:
    AZ_TYPE_INFO(ScriptTimePoint, "{4c0f6ad4-0d4f-4354-ad4a-0c01e948245c}");
    AZ_CLASS_ALLOCATOR(ScriptTimePoint, SystemAllocator, 0);

    ScriptTimePoint()
        : m_timePoint(AZStd::chrono::system_clock::now()) {}
    //...
}
Enter fullscreen mode Exit fullscreen mode

And system_clock::now() in Code\Framework\AzCore\AzCore\std\chrono\clocks.h:

class system_clock
{
public:
    //...
    static time_point now() { return time_point(duration(AZStd::GetTimeNowMicroSecond())); }
    //...
};
Enter fullscreen mode Exit fullscreen mode

All of this is inlined leaving us with no way to correctly initialize ScriptTimePoint in Rust.

There’s a few different ways to deal with this:

extern "C" {
      #[link_name = "\u{1}?now@system_clock@chrono@AZStd@@SA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@23@XZ"]
      pub fn system_clock_now() -> root::AZStd::chrono::system_clock_time_point;
  }
  impl system_clock {
      #[inline]
      pub unsafe fn now() -> root::AZStd::chrono::system_clock_time_point {
          system_clock_now()
      }
  }
Enter fullscreen mode Exit fullscreen mode
  • Wrap the function and export it from C++: 
extern "C" {
      time_point system_clock_now() {
          return system_clock::now();
      }
  }
Enter fullscreen mode Exit fullscreen mode

Then in Rust:

extern "C" {
      pub fn system_clock_now() -> time_point;
  }
Enter fullscreen mode Exit fullscreen mode
  • Re-implement the helper in Rust: 
impl AZStd::chrono::system_clock {
      pub fn now() -> AZStd::chrono::system_clock_time_point {
          unsafe {
              let time = AZStd::GetTimeNowMicroSecond();
              let duration = AZStd::chrono::duration::from_duration_rep(time);
              AZStd::chrono::time_point::from_time_point_duration(duration)
          }
      }
  }

  impl<Rep> AZStd::chrono::duration<Rep> {
      pub fn from_duration_rep(val: Rep) -> Self {
          Self { m_rep: val, _phantom_0: PhantomData }
      }
  }

  impl<Duration> AZStd::chrono::time_point<Duration> {
      pub fn from_time_point_duration(val: Duration) -> Self {
          Self { m_d: val, _phantom_0: PhantomData }
      }
  }
Enter fullscreen mode Exit fullscreen mode

At the moment, some combination of the first two seems like the right choice:

  1. Requires least amount of code to write/debug/maintain
  2. If original C++ code is removed/renamed, we’ll get a compile time error to update the wrapper
  3. Leverage bindgen to automatically generate Rust bindings

Solution

First, we create wrappers in a C++ static library to access from Rust:

// RustAz.h
namespace AZStd {
    namespace chrono {
        system_clock::time_point system_clock_now()
        {
            return system_clock::now();
        }
    }
}
// RustAz.cpp
#include "RustAz.h"
Enter fullscreen mode Exit fullscreen mode

RustAz/wscript to produce RustAz.lib:

def build(bld):
    bld.CryEngineStaticLibrary(
        target = 'RustAz',
        #...
    )
Enter fullscreen mode Exit fullscreen mode

Build, and dumpbin /symbols <root>BinTemp\win_x64_vs2017_profile\Code\Framework\RustAz\RustAz\RustAz.lib confirms it is an externally visible symbol defined in the library:

287 00000000 SECT113 notype () External | 

?system_clock_now@chrono@AZStd@@YA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@12@XZ 

(class AZStd::chrono::time_point<class AZStd::chrono::system_clock,class AZStd::chrono::duration< __int64,class AZStd::ratio<1,1000000> > >__ cdecl AZStd::chrono::system_clock_now(void))
Enter fullscreen mode Exit fullscreen mode

Next, process RustAz.h with bindgen to generate the Rust bindings:

pub mod AZStd {
    pub mod chrono {
        extern "C" {
            #[link_name = "\u{1}?system_clock_now@chrono@AZStd@@YA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@12@XZ"]
            pub fn system_clock_now() -> root::AZStd::chrono::system_clock_time_point;
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Ebus-in’

From Rust we can now make a request via the ebus and receive a simple result:

unsafe {
    use lmbr_sys::root::{AZ, AZStd};
    let mut current_time = AZ::ScriptTimePoint { m_timePoint: AZStd::chrono::system_clock_now() } ;
    lmbr_sys::TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(&mut current_time);
    info!("GetTimeAtCurrentTick {:?}", current_time);
}
Enter fullscreen mode Exit fullscreen mode

I must admit, the prospect of having to re-write and manually maintain extensive bindings is daunting and dissuades me from wanting to continue pursuing this. However, there doesn’t seem to be a better option until bindgen gets support for more than the most trivial C++.

💖 💪 🙅 🚩
jeikabu
jeikabu

Posted on December 6, 2019

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Lumberyard Asset Builder in Rust
lumberyard Lumberyard Asset Builder in Rust

February 21, 2020

Lumberyard EBus from Rust
lumberyard Lumberyard EBus from Rust

December 6, 2019