Creating a meta-object system for C++

A while ago I developed a meta-object system for C++ to use on a game engine I started making, but it is now obsolete due to converting the engine to C# (I’ll make a post about that.. someday). This post was supposed to be the first part of the game engine development posts, but since it’s not part of the engine, it’ll be on it’s own now.

Do keep in mind that this was not designed to be a general-purpose meta-object system. It was designed for a game engine. It’s a relatively simple system, but it is very useful for serialization and class metadata. This is also incomplete, so features such as array property types are not present, and the code is taken from the engine, so it does not work as-is. It is merely a demonstration of how a meta-object system can be done. But hopefully this will be of some use to someone.

Full code

Some of the code is not shown here and most of it is split into parts, but you can inspect the full code here.

Meta-objects?

Say you need to access data and metadata of a C++ class without actually knowing the type itself, during runtime. This mechanism is often referred to as reflection or Run-time type information (RTTI), although RTTI refers to introspection, while reflection refers to both introspection and modification. Unlike in higher-level languages such as C# and Java, reflection is not built-in in C++. And while there are multiple different libraries supporting reflection for C++, many of them are too complex for the needs of a simple property system.

Reflection can be used for serialization and providing additional metadata about classes. In the case of a game engine, we can use reflection to serialize properties of game objects that we add into a scene, and provide human-readable labels for our properties in the SDK. These use cases rely on not having to know the exact type of an object during compile-time; that they can read and modify the object during run-time regardless of its type.

Our reflection system is made up of several parts.

  • WObject. Class that all objects using the reflection system need to inherit.
  • Repository. This is a singleton that contains all meta-objects.
  • Meta-objects. These are used to access the metadata of a certain class.
  • Properties. These are basically object variables that are exposed to the reflection system.
  • Inputs. Object methods that can be invoked via the reflection system.
  • Outputs. Variables of type WOutput. These are not properties because they are treated very differently within the engine.
  • Attributes. These are simple text values that we can attach to a certain type. Can be used to provide additional data about a type.

I will talk about properties, inputs and outputs and how they are used in the game engine series. For now, all you need to know is that properties are variables, inputs are methods and outputs are variables of a specific type, WOutput.

Repository

Let’s start with the easiest class to write; The repository. It is basically a container that can be used to find meta-objects. The meta repository is a singleton, meaning only one instance of the object exists within the program runtime.

repository.h

#ifndef WMETAREPOSITORY_H
#define WMETAREPOSITORY_H

#include <vector>
#include <string>
#include "object.h"
#include "property.h"
#include "input.h"
#include "output.h"
#include "attribute.h"
#include "macros.h"

class WMetaRepository
{
public:
    static WMetaRepository *getInstance();

    WMetaRepository();

    int metaObjectCount() const;
    WMetaObject *metaObject(int index) const;

    WMetaObject *findByClassName(std::string name) const;

    void addMetaObject(WMetaObject *obj);

private:
    static WMetaRepository *sInstance;

    std::vector<WMetaObject*> mMetaObjects;
};

#endif // WMETAREPOSITORY_H

repository.cpp

#include "repository.h"

WMetaRepository *WMetaRepository::sInstance = 0;

WMetaRepository *WMetaRepository::getInstance()
{
    if(!sInstance)
        sInstance = new WMetaRepository();
    return sInstance;
}

WMetaRepository::WMetaRepository()
{

}

int WMetaRepository::metaObjectCount() const
{
    return mMetaObjects.size();
}

WMetaObject *WMetaRepository::metaObject(int index) const
{
    return mMetaObjects[index];
}

WMetaObject *WMetaRepository::findByClassName(std::string name) const
{
    for(int i = 0; i < mMetaObjects.size(); i++) {
        WMetaObject *obj = mMetaObjects[i];
        if(obj->name() == name)
            return obj;
    }
    return 0;
}

void WMetaRepository::addMetaObject(WMetaObject *obj)
{
    if(findByClassName(obj->name())) {
        return;
    }
    mMetaObjects.push_back(obj);
}

We can also add iterator methods to support iterator loops for meta objects, but this will do for now. findByClassName is a method for finding a meta-object by giving it a class name. It is especially useful when deserializing objects, as we save the type of our object as text.

Note that all classes we are going to make are prefixed with W. This is just a naming convention for the game engine. The right thing to do would be to use namespaces, but the classes are a little bit easier to use without namespaces. It’s a design decision, up to you.

MetaObject

Next up, the MetaObject class. We’ll go through the function definitions one by one.

object.h

#ifndef WMETAOBJECT_H
#define WMETAOBJECT_H

#include "../global.h"
#include <string>
#include <vector>

class WMetaInfo;
class WMetaProperty;
class WMetaInput;
class WMetaOutput;
class WMetaAttribute;

/**
 * @brief Provides metadata about a WObject class.
 */
class WMetaObject
{
public:
    typedef void*(*DefaultConstructorFunc)();

    WMetaObject(std::string name, std::string baseClassName, DefaultConstructorFunc defaultConstructor,
                std::vector<WMetaInfo*> info);
    ~WMetaObject();

    std::string name() const;

    std::string baseClassName() const;
    WMetaObject *baseClass() const;

    bool isSubclassOf(std::string type) const;

    void *createInstance();

    std::vector<WMetaProperty*>::const_iterator propertiesBegin() const;
    std::vector<WMetaProperty*>::const_iterator propertiesEnd() const;

    /** Find property meta info with the given name. Searches recursively from base classes as well. */
    WMetaProperty *getProperty(std::string name) const;

    std::vector<WMetaInput*>::const_iterator inputsBegin() const;
    std::vector<WMetaInput*>::const_iterator inputsEnd() const;

    /** Find input meta info with the given name. Searches recursively from base classes as well. */
    WMetaInput *getInput(std::string name);

    std::vector<WMetaOutput*>::const_iterator outputsBegin() const;
    std::vector<WMetaOutput*>::const_iterator outputsEnd() const;

    /** Find output meta info with the given name. Searches recursively from base classes as well. */
    WMetaOutput *getOutput(std::string name);

    std::vector<WMetaAttribute*>::const_iterator attributesBegin() const;
    std::vector<WMetaAttribute*>::const_iterator attributesEnd() const;

    /** Get value of an attribute of given name. NOT recursive. */
    std::string getAttribute(std::string name) const;

private:
    std::string mName;
    std::string mBaseClassName;
    DefaultConstructorFunc mDefaultConstructor;
    std::vector<WMetaProperty*> mProperties;
    std::vector<WMetaInput*> mInputs;
    std::vector<WMetaOutput*> mOutputs;
    std::vector<WMetaAttribute*> mAttributes;
};

class WMetaInfo
{
    public:
        enum Type {
            TYPE_PROPERTY,
            TYPE_INPUT,
            TYPE_OUTPUT,
            TYPE_ATTRIBUTE
        };

        WMetaInfo(Type type) : mInfoType(type), mMetaObject(0) {}

        Type infoType() const
        {
            return mInfoType;
        }

        WMetaObject *metaObject() const
        {
            return mMetaObject;
        }

    private:
        friend class WMetaObject;

        Type mInfoType;
        WMetaObject *mMetaObject;
};

#endif // WMETAOBJECT_H

First, let’s make the constructor.

WMetaObject::WMetaObject(std::string name, std::string baseClassName, DefaultConstructorFunc defaultConstructor,
                         std::vector<WMetaInfo *> info) :
    mName(name), mBaseClassName(baseClassName), mDefaultConstructor(defaultConstructor)
{
    for(int i = 0; i < info.size(); i++) {
        WMetaInfo *inf = info[i];
        inf->mMetaObject = this;
        switch(inf->infoType()) {
        case WMetaInfo::TYPE_PROPERTY:
            mProperties.push_back((WMetaProperty*)inf);
            break;
        case WMetaInfo::TYPE_INPUT:
            mInputs.push_back((WMetaInput*)inf);
            break;
        case WMetaInfo::TYPE_OUTPUT:
            mOutputs.push_back((WMetaOutput*)inf);
            break;
        case WMetaInfo::TYPE_ATTRIBUTE:
            mAttributes.push_back((WMetaAttribute*)inf);
            break;
        }
    }
    WMetaRepository *repo = WMetaRepository::getInstance();
    repo->addMetaObject(this);
}

Here we set the name, default constructor, base class and meta-info of the represented class. DefaultConstructorFunc is defined in the header file, and is simply a function pointer to a void function with no parameters. We store the base class as a string so that we can later use the repository to search for the meta-object of our base class. Meta-info is a collection of metadata attached to our meta-object, which includes properties, inputs, outputs and attributes.

The constructor adds all meta-info to their specified containers and adds the meta-object to the repository. The reason we need to add it in the constructor is that we need to create the meta-object outside function scope, and we only have a limited amount of statements we can do outside functions. Some may call this a dirty method, but it’s better than registering all of our metadata inside our main function.

Next, we have the destructor.

WMetaObject::~WMetaObject()
{
    for(auto it = mProperties.begin(); it != mProperties.end(); ++it) {
        if(*it) {
            delete *it;
        }
    }
    for(auto it = mInputs.begin(); it != mInputs.end(); ++it) {
        if(*it) {
            delete *it;
        }
    }
    for(auto it = mOutputs.begin(); it != mOutputs.end(); ++it) {
        if(*it) {
            delete *it;
        }
    }
    for(auto it = mAttributes.begin(); it != mAttributes.end(); ++it) {
        if(*it) {
            delete *it;
        }
    }
}

This is quite straightforward; we delete all meta-info as they are heap-allocated objects.

WMetaObject *WMetaObject::baseClass() const
{
    WMetaRepository *repo = WMetaRepository::getInstance();
    for(int i = 0; i < repo->metaObjectCount(); i++) {
        WMetaObject *obj = repo->metaObject(i);
        if(obj->name() == mBaseClassName)
            return obj;
    }
    return 0;
}

bool WMetaObject::isSubclassOf(std::string type) const
{
    if(name() == type)
        return true;

    WMetaObject *bMeta = baseClass();
    while(true) {
        if(!bMeta)
            return false;
        if(bMeta->name() == type)
            return true;
        bMeta = bMeta->baseClass();
    }
}

These are utility functions for easily accessing class hierarchy from the meta-object.

WMetaProperty *WMetaObject::getProperty(std::string name) const
{
    for(int i = 0; i < mProperties.size(); i++) {
        WMetaProperty *prop = mProperties[i];
        if(prop->name() == name)
            return prop;
    }
    WMetaObject *b = baseClass();
    if(b)
        return b->getProperty(name);
    else
        return 0;
}

WMetaInput *WMetaObject::getInput(std::string name)
{
    for(int i = 0; i < mInputs.size(); i++) {
        WMetaInput *inp = mInputs[i];
        if(inp->name() == name)
            return inp;
    }
    WMetaObject *b = baseClass();
    if(b)
        return b->getInput(name);
    else
        return 0;
}

WMetaOutput *WMetaObject::getOutput(std::string name)
{
    for(int i = 0; i < mOutputs.size(); i++) {
        WMetaOutput *outp = mOutputs[i];
        if(outp->name() == name)
            return outp;
    }
    WMetaObject *b = baseClass();
    if(b)
        return b->getOutput(name);
    else
        return 0;
}

std::string WMetaObject::getAttribute(std::string name) const
{
    for(auto it = mAttributes.begin(); it != mAttributes.end(); ++it) {
        WMetaAttribute *att = *it;
        if(att->name() == name)
            return att->value();
    }
    return "";
}

These methods can be used to recursively search for different kinds of meta-info from the meta-object by name.

void *WMetaObject::createInstance()
{
    return mDefaultConstructor();
}

This can be used to create an instance of the object into heap-allocated memory. It simply calls the function pointer that was passed to the meta-object in the constructor.

The rest are quite self-explanatory methods that include getters and iterator getters.

std::string WMetaObject::name() const
{
    return mName;
}

std::string WMetaObject::baseClassName() const
{
    return mBaseClassName;
}

std::vector<WMetaProperty*>::const_iterator WMetaObject::propertiesBegin() const
{
    return mProperties.begin();
}

std::vector<WMetaProperty*>::const_iterator WMetaObject::propertiesEnd() const
{
    return mProperties.end();
}

std::vector<WMetaInput*>::const_iterator WMetaObject::inputsBegin() const
{
    return mInputs.begin();
}

std::vector<WMetaInput*>::const_iterator WMetaObject::inputsEnd() const
{
    return mInputs.end();
}

std::vector<WMetaOutput*>::const_iterator WMetaObject::outputsBegin() const
{
    return mOutputs.begin();
}

std::vector<WMetaOutput*>::const_iterator WMetaObject::outputsEnd() const
{
    return mOutputs.end();
}

MetaProperty

Now we’re getting to the more interesting parts of accessing unknown object types.

property.h

#ifndef WMETAPROPERTY_H
#define WMETAPROPERTY_H

#include <string>
#include "object.h"

enum WMetaPropertyType
{
    PROPERTY_BOOL,

    PROPERTY_FLOAT,
    PROPERTY_DOUBLE,

    PROPERTY_INT8,
    PROPERTY_INT16,
    PROPERTY_INT32,
    PROPERTY_INT64,

    PROPERTY_UINT8,
    PROPERTY_UINT16,
    PROPERTY_UINT32,
    PROPERTY_UINT64,

    PROPERTY_STRING,

    PROPERTY_VECTOR,
    PROPERTY_ANGLE,
    PROPERTY_QUATERNION,
    PROPERTY_TRANSFORM,
    PROPERTY_MATRIX,
    PROPERTY_COLOR,

    PROPERTY_UID,

    PROPERTY_OBJECT,

    // Special property type, with very limited use
    PROPERTY_CUSTOM
};

enum WMetaPropertyFlags
{
    PFLAGS_NONE = 0, // No flags
    PFLAGS_EDITABLE = (1 << 0), // Property is editable within the SDK
    PFLAGS_ARRAY = (1 << 1), // Property is an array of the given type, and not a single instance
    PFLAGS_SERIALIZEABLE = (1 << 2), // Property can be serialized
};

class WINDENGINE_DLL WMetaProperty : public WMetaInfo
{
    public:
        WMetaProperty(std::string name, WMetaPropertyType type, int flags, int offset, int size);
        WMetaProperty(std::string name, std::string objectType, int flags, int offset, int size);

        std::string name() const;
        WMetaPropertyType type() const;

        /**
         * @brief Return object type name. Only useful if property type is OBJECT
         * @return
         */
        std::string objectType() const;

        int flags() const;

        int offset() const;
        int size() const;

        void getValue(void *obj, void *buf);
        void setValue(void *obj, void *buf);

    private:
        std::string mName;
        WMetaPropertyType mType;
        std::string mObjectType;
        int mFlags;
        int mOffset;
        int mSize;
};

#endif // WMETAPROPERTY_H

Take note of the enumeration types. WMetaPropertyType defines all different types of properties that we can have. It includes all basic types, strings and custom structures like vectors, transforms and matrices, which we’ll be going through in another part of the game engine series. Basically, you add all the different data types that you want to support here. WMetaPropertyFlags define other attributes that we may want to define for a property, such as whether the property is editable within the SDK, whether it is serializable, and whether it is an array instead of a single value.

void WMetaProperty::getValue(void *obj, void *buf)
{
    switch(mType) {
    case PROPERTY_STRING:
    {
      std::string *str = (std::string*)buf;
      std::string *objStr = (std::string*)((char*)obj + mOffset);
      *str = *objStr;
      break;
    }
    //case PROPERTY_ENTITY:
    //case PROPERTY_COMPONENT:
    //case PROPERTY_ASSET:
    case PROPERTY_OBJECT:
    {
      WPointerBase *objSp = (WPointerBase*)((char*)obj + mOffset);
      WPointerBase *sp = (WPointerBase*)buf;
      sp->set(objSp->rawPointer());
      break;
    }
    default:
      memcpy(buf, (char*)obj + mOffset, mSize);
      break;
    }
}

void WMetaProperty::setValue(void *obj, void *buf)
{
    switch(mType) {
    case PROPERTY_STRING:
    {
      std::string *str = (std::string*)buf;
      std::string *objStr = (std::string*)((char*)obj + mOffset);
      *objStr = *str;
      break;
    }
    //case PROPERTY_ENTITY:
    //case PROPERTY_COMPONENT:
    //case PROPERTY_ASSET:
    case PROPERTY_OBJECT:
    {
      WPointerBase *objSp = (WPointerBase*)((char*)obj + mOffset);
      WPointerBase *sp = (WPointerBase*)buf;
      objSp->set(sp->rawPointer());
      break;
    }
    default:
      memcpy((char*)obj + mOffset, buf, mSize);
      break;
    }
}

This is where the magic of anonymously reading properties is. In the default switch case, which is used for most property types, the C method memcpy is used. It takes two pointers and one integer as arguments, being the source address, destination address and data size in bytes, respectively. So we are essentially copying raw data from one part of the memory to another. We know the offset of the property and it’s size in bytes, so we can take the object’s pointer, convert it to a char pointer and add the offset, giving us the memory address of the property.

However, reading a property like this assumes that the property is entirely stored on the object’s data structure. This is not always the case. For example, with the std::string type, the actual text is not stored within the object itself, it merely contains utility methods and a pointer to actual string. It’s essentially a C-string wrapper. So when memcpy is used to copy the value, we now have to strings pointing to the same C-string, which is not expected behavior for std::string. Strings delete their C-strings when their destructor is called, which will cause access violation if two strings point to the same C-string. The object type is also assumed to be a smart pointer type, which also cannot be raw copied.

#include "property.h"
#include "../util/pointer.h"
#include <cstring>
#include <memory>

WMetaProperty::WMetaProperty(std::string name, WMetaPropertyType type, int flags, int offset, int size)
    : WMetaInfo(TYPE_PROPERTY), mName(name), mType(type), mFlags(flags), mOffset(offset), mSize(size)
{
}

WMetaProperty::WMetaProperty(std::string name, std::string objectType, int flags, int offset, int size)
    : WMetaInfo(TYPE_PROPERTY), mName(name), mType(PROPERTY_OBJECT), mObjectType(objectType), mFlags(flags), mOffset(offset), mSize(size)
{

}

std::string WMetaProperty::name() const
{
    return mName;
}

WMetaPropertyType WMetaProperty::type() const
{
    return mType;
}

std::string WMetaProperty::objectType() const
{
    return mObjectType;
}

int WMetaProperty::flags() const
{
    return mFlags;
}

int WMetaProperty::offset() const
{
    return mOffset;
}

int WMetaProperty::size() const
{
    return mSize;
}

This is the rest of the property.cpp file. Just constructors and getters.

MetaInput, MetaOutput

Next we define two classes that hold metadata about inputs and outputs, which is a concept of event-driven functionality for the engine, similar to Source engine. An input is like a method that can be invoked, and all outputs connected to that input will be signaled that the input was called. Ideally, you would be able to link outputs to inputs both in code and in editor.

input.h

#ifndef WMETAINPUT_H
#define WMETAINPUT_H

#include "object.h"
#include "property.h"

class WINDENGINE_DLL WMetaInput : public WMetaInfo
{
    public:
        WMetaInput(std::string name, std::vector<WMetaPropertyType> parameterTypes);

        std::string name() const;

        int parameterCount() const;
        WMetaPropertyType getParameterType(int index) const;

        virtual void invoke(void *obj, void *p1 = 0) = 0;

    private:
        std::string mName;
        std::vector<WMetaPropertyType> mParameterTypes;
};

template<class C>
class WMetaInputNoParams : public WMetaInput
{
    public:
        typedef void(C::*FuncP)();

        WMetaInputNoParams(std::string name, std::vector<WMetaPropertyType> parameterTypes, FuncP methodPointer)
            : WMetaInput(name, parameterTypes), mMethodPointer(methodPointer)
        {
        }

        void invoke(void *obj, void *p1 = 0)
        {
            (((C*)obj)->*mMethodPointer)();
        }

    private:
        FuncP mMethodPointer;
};

template<class C, class T>
class WMetaInputOneParam : public WMetaInput
{
    public:
        typedef void(C::*FuncP)(T);

        WMetaInputOneParam(std::string name, std::vector<WMetaPropertyType> parameterTypes, FuncP methodPointer)
            : WMetaInput(name, parameterTypes), mMethodPointer(methodPointer)
        {
        }

        void invoke(void *obj, void *p1 = 0)
        {
            (((C*)obj)->*mMethodPointer)(*((T*)p1));
        }

    private:
        FuncP mMethodPointer;
};

#endif // WMETAINPUT_H

Here we have three classes; WMetaInput, WMetaInputNoParams and WMetaInputOneParam. A subclass of WMetaInput needs to be created for a number of parameters an input method can take, because template types need to be utilized for each parameter. More subclasses can be made, but I personally didn’t need more than one parameter for inputs.

Let’s look at the line of code that might’ve caught your eye.

(((C*)obj)->*mMethodPointer)(*((T*)p1));

What we’re basically doing here is invoking a method of obj, identified by mMethodPointer. obj and p1 are void pointers, while mMethodPointer is a function pointer. obj is casted to the pointer type that this input method is for, and p1 is first casted to the second template type and then dereferenced, invoking the method with the same value that p1 is pointing to.

input.cpp only defines getters and setters that were not defined in the header file.

output.h

#ifndef WMETAOUTPUT_H
#define WMETAOUTPUT_H

#include "object.h"
#include "property.h"

class WOutput;

class WINDENGINE_DLL WMetaOutput : public WMetaInfo
{
    public:
        WMetaOutput(std::string name, std::vector<WMetaPropertyType> parameterTypes, int offset);

        std::string name() const;
        int offset() const;

        int parameterCount() const;
        WMetaPropertyType getParameterType(int index) const;

        WOutput *getReference(void *obj) const;

    private:
        std::string mName;
        std::vector<WMetaPropertyType> mParameterTypes;
        int mOffset;
};

#endif // WMETAOUTPUT_H

output.cpp

#include "output.h"
#include "../events/woutput.h"

WMetaOutput::WMetaOutput(std::string name, std::vector<WMetaPropertyType> parameterTypes, int offset)
    : WMetaInfo(TYPE_OUTPUT), mName(name), mParameterTypes(parameterTypes), mOffset(offset)
{
}

std::string WMetaOutput::name() const
{
    return mName;
}

int WMetaOutput::offset() const
{
    return mOffset;
}

int WMetaOutput::parameterCount() const
{
    return mParameterTypes.size();
}

WMetaPropertyType WMetaOutput::getParameterType(int index) const
{
    return mParameterTypes[index];
}

WOutput *WMetaOutput::getReference(void *obj) const
{
    return (WOutput*)((char*)obj + mOffset);
}

WMetaOutput is very similar to WMetaProperty, since it refers to an object variable of type WOutput, so for example, getReference is a similar method to WMetaProperty‘s getValue.

WMetaAttribute in attribute.h defines metadata for attributes, which are basically static data attached to classes. For example, components can use requiredComponent attribute to describe what other components that component depends on.

Macros

Finally, we define some macros to make the use of this meta-object system less tedious.

macros.h

#ifndef WMETAMACROS_H
#define WMETAMACROS_H

#define META_OBJECT_INIT(C, BC) \
    public: \
        typedef BC BaseClass; \
        virtual WMetaObject *metaObject() const; \
        static WMetaObject *staticMetaObject(); \
        static std::vector<WMetaInfo*> getMetaInfo(); \
    private: \
        static WMetaObject sMetaObject;

#define META_OBJECT_REGISTER(C, BC) \
    WMetaObject *C::metaObject() const { \
        return &C::sMetaObject; \
    } \
    WMetaObject *C::staticMetaObject() { \
        return &sMetaObject; \
    } \
    static void *getDefaultConstructorOf##C() { \
        return new C(); \
    } \
    WMetaObject C::sMetaObject(#C, #BC, &getDefaultConstructorOf##C, C::getMetaInfo());

#define META_INFO_BEGIN(C) \
    std::vector<WMetaInfo*> C::getMetaInfo() { \
        typedef C ThisClass; \
        C inst; \
        std::vector<WMetaInfo*> metaInfo; \

#define META_INFO_END() \
        return metaInfo; \
    }

#define META_PROPERTY(Name, Type, Flags) \
    metaInfo.push_back(new WMetaProperty(#Name, Type, Flags, \
    (char*)&inst.Name - (char*)&inst, sizeof(inst.Name)));

#define META_PROPERTY_OBJECT(Name, Flags) \
    metaInfo.push_back(new WMetaProperty(#Name, inst.Name.type(), Flags, \
    (char*)&inst.Name - (char*)&inst, sizeof(inst.Name)));

#define META_INPUT_NO_PARAMS(Name) \
    std::vector<WMetaPropertyType> pTypesFor##Name; \
    metaInfo.push_back(new WMetaInputNoParams<ThisClass>(#Name, \
        pTypesFor##Name, &ThisClass::Name));

#define META_INPUT_ONE_PARAM(Name, Type1, ActualType1) \
    std::vector<WMetaPropertyType> pTypesFor##Name; \
    pTypesFor##Name.push_back(Type1); \
    metaInfo.push_back(new WMetaInputOneParam<ThisClass, ActualType1>(#Name, \
        pTypesFor##Name, &ThisClass::Name));

#define META_OUTPUT_NO_PARAMS(Name) \
    std::vector<WMetaPropertyType> pTypesFor##Name; \
    metaInfo.push_back(new WMetaOutput(#Name, pTypesFor##Name, (char*)&inst.Name - (char*)&inst));

#define META_OUTPUT_ONE_PARAM(Name, Type1) \
    std::vector<WMetaPropertyType> pTypesFor##Name; \
    pTypesFor##Name.push_back(Type1); \
    metaInfo.push_back(new WMetaOutput(#Name, pTypesFor##Name, (char*)&inst.Name - (char*)&inst));

#define META_ATTRIBUTE(Name, Value) \
    metaInfo.push_back(new WMetaAttribute(#Name, Value));

#endif // WMETAMACROS_H

Conclusion

Honestly, this blog post was kind of rushed in the end. I started writing this when I was making the meta-object system, and it kind of stopped when I moved to my C# solution. Then summer came, and I pretty much forgot about it. But it was halfway done, so I decided to finish it anyway, hoping it will be useful to someone.

I won’t stop blogging entirely, but I’m not going to be making posts very often. I only have a few interesting topics I want to write about, and making those posts will take time.

Thanks for reading. Feel free to comment about typos or suggestions that I could fix or add into this post, if you have any.

First VR game development

Last thursday, I participated in a competitive 12-hour game jam. Before the jam, since we had a chance to use VR devices like Oculus and Myo, we thought “Hey, let’s make a VR game in 12 hours with zero prior experience in VR game design or development!”. Our goal wasn’t really to make a good game or win the competition, we primarily wanted to get a grasp of what it’s like to develop VR games. But suffice it to say, we didn’t manage to make a “complete” game, even within the standards of a game jam.

The concept

What we were supposed to make was a different kind of castle defense game, trying to integrate the VR devices into gameplay. You are in a defensive position of a castle and your goal is to keep firing cannons to keep enemies at bay, and to prevent your dumb assistants from adding too much gunpowder into the cannons. Your goal is to defend the castle as long as possible, and avoid igniting cannons that have too much gunpowder. In either case, you will lose, and your score is based on the amount of enemies destroyed.

The player controls a first-person character with WASD movement, the Myo controller and Oculus. You use Myo to control your hand that holds a torch, and Oculus to look around, of course. You light your torch with a campfire and start igniting the cannons. If a cannon doesn’t have enough gunpowder when fired, it will not reach the target, and you get no points.

So the concept heavily revolves around the devices that we were using. Without them, it would’ve probably ended up as a bad copy of Surgeon Simulator.

Production

The plan of our production was to get a prototype of our core idea done in about 2 hours into the jam. So we spent the next 2 hours doing anything but the prototype, including trying to get the devices work properly.

About 4 hours in, we ditched Myo. It was buggy and quite inaccurate. We changed the controls so that you can control the torch arm with a mouse.

After we got the new controls somewhat working, we ate and then we had about 6 hours left and even the first prototype wasn’t ready. At this point we decided we are going forward with our current idea until the end and see how much we can get done.

One hour until deadline, we started adding the score system and game loss logic so that we can actually call it a game and not a toy, and started building. The end result looked about as good as most kickstarter projects.

Things that could’ve gone better

I’m just going to list these because it’s probably easier to read that way.

  • Planning ahead. We knew the jam’s theme beforehand, so we could’ve thought about the concept more and get straight to prototype development once the jamming started. We also should’ve arrived sooner to get the VR devices working properly.
  • Ditch Myo sooner. Or, in other words, we should’ve failed faster. We should’ve prototyped faster to quickly realize that Myo isn’t going to work for our game.
  • Take time into consideration. The part that the game jam was only 12 hours didn’t really sink in at the beginning. 12 hours is a very short time for a game jam. But we knew that we had 12 hours right when we signed up for the jam. We could’ve allocated our time better.
  • Juiciness. Even though we didn’t have a fully working game, we could’ve made it more juicy so that it would be more interesting as we presented it.
  • Graphics. This is a minor thing, though. We didn’t really have an artist in our team so we can scrub this off with “It’s engineer graphics, don’t question it.”

Conclusion

This blog post may make the jam seem like a negative experience, but it definitely was not. Sure, our game kind of sucked, but we still made our first VR game. We got a nice concept that we may someday properly produce. We got to play around with Oculus, which is always nice. We got free pizza. And despite the occasional stressing (at least on my part), we had fun, which in itself signals a successful game jam.

Maybe we’ll do better in the next competitive jam, whenever that’s going to be. 🙂

Getting better at jamming in Global Game Jam 2016

So the last weekend of January was the weekend of Global Game Jam and I participated for the second time now. This time was quite different though; since the GGJ site in Kajaani was held mostly for the students in our school, I wasn’t making a game with complete strangers. Me and five of my schoolmates made this. As you can see from the result, we definitely had a fun and non-serious jam, which is what I believe game jams should be. Innovation is also a plus, but the innovativeness of our game is… debatable.

I also tried to learn something from my previous experience in Global Game Jam and try doing things a bit differently. And I might have actually learned something. Here’s some advice that you might or might have not heard before.

Organizing features

There isn’t much time to design the game you are making during a jam, so being short and concise is important. An easy way to map out the game is to list it’s features. But an even better way is to categorize the features into at least two categories: core and extended. Core features contain the minimum of what is required for an enjoyable game. So focusing and perfecting the core features first before extended features improve your chances of making a good game. Like in our game, if we had focused on something trivial like a proper score system, we would devote time from more important things like level pacing, jump mechanics and the juiciness of them.

However, we could’ve done the organizing a bit better. We made a level generator system that was supposed to make it easier to add obstacles in the music’s rhythm. Thinking back, this was a pretty weird decision. First of all, we allocated time to make a tool and not a game. Second of all, we had to allocate time to introduce it to the level designers. Third of all, it didn’t really make level designing that much faster. What we could’ve done instead is forget about making the level generator, let the level designers work with standard Unity scene editor and focus our programming efforts on something more important.

Teamwork

Saying that working in a team requires teamwork is obvious, but it’s not always as easy as it sounds. Some people don’t get along with each other and some people have difficulty working together, whether it’s because of communication issues or personal issues. I don’t think I’ve experienced much of the latter, but the former certainly does happen. I occasionally have difficulties getting my ideas across to other people, and I’m pretty sure I’m not the only one. It’s good to keep this in mind, and ask your team members to clarify if you don’t understand everything. Not fully understanding your team members can lead to mistakes that take too much time to fix later on.

Also, even if your team members are all specialized in some aspect of the game (programming, graphics, level design..), they can still do other things as well. And this might be necessary, given the limited time you have. And it’s especially useful when some of your team members have nothing else to do anymore.

Take breaks

A weekend jam is a pretty intensive weekend, and you’re probably going to be more productive than your entire last month. At least if you’re not employed. So taking regular breaks is important. Take a cup of coffee, dump your mind by drawing stuff on the whiteboard and go bother other teams with questions like “How’s your game coming along?”. Either way, taking a short break at least once an hour can do wonders for both productivity and your mental state.

Sleep

Just do. Sleep deprivation can make jamming more stressing, less productive and most importantly, less fun. And the only way to avoid sleep deprivation is to actually sleep. Coffee just postpones the inevitable.

This is probably one of the most difficult advises to follow, for me at least. But us humans need sleep, so we must sleep.

And have fun

So in a nutshell; organize, work with your team, take breaks, sleep and most importantly, have fun.

Also, if you’re anxious of participating in GGJ, whether you think you are not social enough or that you lack skills, just go and participate. Other people with the same exact worries are also joining. Face your fears. Just do it.

Until next time.

Learning Unreal Engine 4: Conway’s Game of Life

So recently I’ve been learning Unreal Engine, both with blueprints and with C++. I decided to implement Conway’s Game of Life in Unreal Engine as a project, because I’ve wanted to do my own implementation for a while, and it seemed like a good way to get started with UE4.

While you’re reading this, you can download the executable and try it out for yourself. Note that the more cells you have in the grid, the more memory and render time it takes, so you might not want to spawn thousands of cells right away.

I’ve heard a lot about UE4 and people seem to like it a lot. It is also a strong competitor against Unity, and with good reason. Unreal Engine has been popular among AAA studios for a long time now, but a few years ago, with the release of UE4, it is gaining popularity among indie developers. Through the Blueprints visual scripting system, it is much more beginner-friendly for programmers as well as level designers than it was before.

I will be comparing UE4 a lot with Unity in this post, since I have more experience in Unity and that previous experience has helped me to migrate knowledge into UE4. The cube mesh used in the simulation is from the standard puzzle example.

Learning the engine structure

The first thing I started to look into was the engine structure differences between UE4 and Unity.

Unity’s base structure is comprised of modules, assets, game objects and components. Modules are how the engine divides it’s functionality into different categories (input, graphics, audio..), assets are files such as data files and textures, game objects are objects in the game world and components are basically what make game objects do something. Each game object must have a Transform component, which stores the object’s position relative to the object’s parent object, or global position if it does not have a parent. For someone who hasn’t used Unity, this structure may seem a bit complex, but it’s much more simple than that of UE4.

UE4 consists of modules, actors, components, controllers, pawns and game modes. Modules have the same idea as in Unity, they are used to categorize different parts of the engine. Your own C++ code will also reside in a module. Actors are all objects that reside in the game world. Unlike in Unity, actors are not only holders for components, they can be inherited, and they can have their own properties. An actor can have multiple components of the same type, and scene components can be stacked in a hierarchy, so they also function differently to that of Unity. Actors also need at least one scene component, but this component is created if it is not present. Controllers are an interface between pawns, players and AI. Pawns are controllable objects in the game world, either controlled by a player or by an AI. Game mode decides what object types to use as pawns, player controllers, spectator pawns, and so on.

So UE4 has a lot more parts in it that you need to get familiar with. But complexity in this case is rarely a bad thing. In Unity, you have to do a lot of things from scratch, or download/buy them from the Asset Store. The complexity of UE4 is, in most cases, useful, and makes your game more structured and easier to maintain.

Of course, the structure of both engines are not really as simple as stated above, that was just an oversimplified comparison. I suggest looking into the documentation of both engines if you want to find out more. I will probably talk more about those on game engine development posts.

Blueprints

blueprints

I was very skeptical about blueprints when I started learning UE4. I’ve always thought of visual scripting as a tool only useful for the basics of learning how to program, and as a useless gimmick in practical use. However, Unreal’s blueprints are a much more sophisticated visual scripting system than anything else I’ve seen before. So I decided to go with it for a while. I thought it was a good way to get to know how the native object types work and how actors interact with each other in the game world.

All was good for a while, although getting used to dragging nodes around was still a bit tedious. That was until I started making the Game of Life simulation logic. Here is a picture of it, in all it’s glory. (click to enlarge)

dontdothis

This is just the basic logic of counting the amount of living neighbors a cell has. Not only is it difficult to read without zooming and dragging all around the place, it was also slow in performance. The game froze even with a 20×20 grid.

So at this point I came to the conclusion that while blueprints are sophisticated and can do a lot of things, they still aren’t a good option for programming more complex functionality. They are slow and messy. They are more useful in adding some small level-specific features to an object, and also UMG, which I will talk about later.

Then I started migrating the project into C++.

C++

I have to say that UE developers made the right choice when sticking with C++ as an option, and adding features such as hot reloading to make rapid prototyping with C++ much more enjoyable. While I miss the simple clutter-less C# code, I don’t mind working with C++ at all, especially since Unreal has a lot of functionality ready to be used.

The first thing I wanted to figure out was how to use Qt Creator with UE. The reason for that is because I absolutely love Qt Creator, and think it’s a far better IDE for C++ than Visual Studio is. It’s faster, more lightweight, cross-platform, somewhat portable and I’ve used it a lot when developing Qt applications. Thankfully, I found a solution, and I didn’t need to use Visual Studio. Yay!

Reflection

cpp_pawn

The Unreal Property System is UE’s reflection system. A reflection system basically gives you the ability to inspect C++ objects, methods and fields during runtime. Like the program is looking at a reflection of itself. This system allows UE to instantiate objects into the game world when it reads the level assets or blueprints. It also allows Unreal Editor to see what kind of classes exist within the C++ codebase, so it can associate them with blueprints and levels.

So the property system is an integral part of UE development, but when creating new object types, you don’t need to do much to make it work. You simply include an automatically generated header file, add a couple of macros (UCLASS, UPROPERTY, UFUNCTION, etc) and let the Unreal Header Tool do the heavy lifting for you. C# has a built-in reflection system, but then again, C++ is a lower-level language.

Input system

The input system in UE is very different from Unity. In Unity, you access the Input static class to get the status of different inputs each frame, where as in UE you bind methods into an InputComponent that will detect changes to that specific input. Unity only has input axes which are used for both buttons and analog input, while UE splits these two into actions and axes, which makes more sense given the input binding mechanic. The pawn class even provides a useful overrideable method for binding input into the player’s pawn.

void AGOLPawn::SetupPlayerInputComponent(UInputComponent* InputComponent)
{
    Super::SetupPlayerInputComponent(InputComponent);

    InputComponent->BindAction("ChangeCameraMode", IE_Pressed, this, &AGOLPawn::ChangeCameraMode);

    InputComponent->BindAction("MoveCamera", IE_Pressed, this, &AGOLPawn::MoveCameraBegin);
    InputComponent->BindAction("MoveCamera", IE_Released, this, &AGOLPawn::MoveCameraEnd);

    InputComponent->BindAction("MoveCameraFurther", IE_Pressed, this, &AGOLPawn::MoveCameraFurther);
    InputComponent->BindAction("MoveCameraCloser", IE_Pressed, this, &AGOLPawn::MoveCameraCloser);

    InputComponent->BindAction("MoveFaster", IE_Pressed, this, &AGOLPawn::MoveFaster);
    InputComponent->BindAction("MoveFaster", IE_Released, this, &AGOLPawn::MoveSlower);

    InputComponent->BindAction("ToggleSimulation", IE_Pressed, this, &AGOLPawn::ToggleSimulation);

    InputComponent->BindAction("ActivateCell", IE_Pressed, this, &AGOLPawn::ActivateCell);
    InputComponent->BindAction("ActivateCell", IE_Released, this, &AGOLPawn::ClearActivationStatus);
    InputComponent->BindAction("DeactivateCell", IE_Pressed, this, &AGOLPawn::DeactivateCell);
    InputComponent->BindAction("DeactivateCell", IE_Released, this, &AGOLPawn::ClearActivationStatus);

    InputComponent->BindAxis("MoveCameraVertical", this, &AGOLPawn::MoveVertical);
    InputComponent->BindAxis("MoveCameraHorizontal", this, &AGOLPawn::MoveHorizontal);

    InputComponent->BindAxis("RotateCameraVertical", this, &AGOLPawn::RotateVertical);
    InputComponent->BindAxis("RotateCameraHorizontal", this, &AGOLPawn::RotateHorizontal);
}

I personally like UE’s input system over Unity. It’s much better structured and you don’t have to put all your input logic on a tick function. You also don’t have to think about where to put your input logic; pawn is your character, so you process input for that character. I reckon that makes multiplayer game code much easier to manage.

Game of Life functionality

Creating a custom pawn for camera controls was a trivial task; bind input, add speed properties, add lock/unlock logic and that was pretty much it. Simple and functional.

The Game of Life functionality could’ve been more simpler but I wanted to focus more on performance. Whenever I needed to query the value of a cell by it’s X and Y position, I could’ve made a method such as this:

bool ALifeSimulator::GetCellValue(int32 gridX, int32 gridY)
{
    for(int32 i = 0; i < mCells.Num(); i++) {
        ALifeCell *cell = mCells[i];
        if(cell->GridX == gridX && cell->GridY == gridY)
            return cell->Value;
    }
    return false;
}

However, this is slower than getting the cell value from a two-dimensional boolean array, like this:

bool ALifeSimulator::GetCellValue(int32 gridX, int32 gridY)
{
    if (!mGrid) return false;
    if (gridX > mSize || gridY > mSize ||
        gridX < 1 || gridY < 1)
        return false;

    return mGrid[gridX - 1][gridY - 1];
}

Not only that, but having cell values in an array makes it faster to copy it during a simulation tick to apply the next state as the current state

for (int x = 1; x <= mSize; x++)
    for (int y = 1; y <= mSize; y++) {
        mGrid[x - 1][y - 1] = mGridNextState[x - 1][y - 1];
    }

Only after the next grid state is ready, we apply the values to the real cell actors

for(int i = 0; i < mCells.Num(); i++) {
    ALifeCell *cell = mCells[i];
    bool value = mGrid[cell->GridX - 1][cell->GridY - 1];
    cell->SetHighlighted(value);
}

So now the only performance limitation we have is how much pretty lights and cube meshes UE4 and the hardware can render, as the two-dimensional array can process thousands of cells with a single tick.

Unreal Motion Graphics

Now that the base game was complete, I wanted to create a nice little user interface for the simulation. Unreal Motion Graphics (UMG) is the de facto standard for creating UI in UE4. UMG is done a bit better than the UI system in Unity. Because of Unity’s simpler engine model, the UI also exists within the game world, which can become a bit tedious when working with it. UMG is not part of the game world; it is added to the local player’s viewport. It can interact with the game world, but it’s not part of it. And this is fine, because it makes little sense to have the UI system within the game world, right?

I used blueprint scripting in UMG, since the logic wasn’t very complex, and it was mostly used for calling C++ functions on actors anyway. Here’s an example:

umg_blueprint

You can code UI elements with C++ as well, so same as actors, don’t stay on blueprints if your logic gets too complex.

Creating standalone releases

After finishing the UI, it was time to pack up the game into a standalone executable. This worked without problems for exporting to Windows 32-bit and 64-bit, but you need to setup a cross-compiler for packaging to any other platform. This is because you need to compile your C++ code to that specific target platform’s native binary format. This probably wouldn’t take more than an hour to set up, but I got lazy at this point, so I only compiled my project for Windows.

Conclusion

After a couple days of intensive jamming, the game simulator was complete! It’s a pretty basic example of an Unreal Engine project, but it was a useful learning experience.

If you missed it, you can download the executable on the top of the post, or just get it here. 🙂

Also, if you want to read more about Conway’s Game of Life, it has an entire wiki dedicated to it.

Introduction

So, this will be the first post on this blog. Now that I finally got around and made a blog, I don’t know what else to put here right now so I’m just gonna talk about my background a bit.

Starting with Garry’s Mod

I have been doing programming as a hobby for some time now. At the beginning of 2010 I started to look into source engine modding out of curiosity, and entered the magical world of C++. Because Valve’s documentation clearly stated that you need to know C++ before doing source mods, I started with some basic hello world applications. I didn’t even get to the source modding part in the end, as my interest shifted towards Lua coding in Garry’s Mod. Lua coding was much more beginner-friendly, and you could do a lot of stuff with it in Gmod, so I pretty much left C++ coding altogether.

In 2011, I also developed interest in making stop motion videos in Garry’s Mod. I did get some following on these videos, but the biggest projects that came out of these was two tools, Stop Motion Helper and Ragdoll Mover. These were my biggest and first public coding projects I did. Years later, these tools are still used by a suprisingly high amount of users. I still occasionally work on these projects, and released a whole new version of Stop Motion Helper about a year ago.

C# and Web Development

At the end of 2012 I got introduced into C# when I stumbled upon the SteamKit2 library, allowing the creation of standalone applications communicating with the Steam network. This got me interested into making a chat bot for Steam. I learned a lot of C# and database programming during this project, as I wanted to create a functional chat log and perform statistics on users and chat messages.

Through database programming, I moved to learn web development, both backend and frontend. I mostly did small website projects, but one project I can recall is VhostManager, which also became my thesis for vocational school. I learned a lot of HTML, CSS, JavaScript, PHP and Python from these projects.

Game development

In the summer of 2013, I got interested in game development. Well, I was interested before, but now I actually started learning. I discovered Unity, and pretty much went with it for a long time, since I already knew C# and Unity was easy to learn and use AND it had the best support for rapid prototyping.

Six months went by, and I had made a dozen prototype projects that never saw the light of day, but I still had a lot of fun doing them. At the time, I had zero experience with collaboration, all my previous projects I had done alone. So in January 2014, I participated in Global Game Jam. It was a lot of fun working with like-minded people, and we even managed to make this thing.

During a game programming course in summer 2014 I started developing a game in Unity that turned out to be my first official release. After a lot of learning, mistakes and procrastination, I released NumberShock in December 2014. With help from DasBoSchitt, the game also got some exposure, with a total of over 5000 downloads as of today. I got a lot of positive feedback, but the game quickly faded away, and not many are playing the game anymore.
Mistakes were made, mistakes were learned from, expectations were crushed, crushed expectations were accepted. All in all, I accept it as my first official game release.

Conscription

So 2015 was a very busy year in my life. In January 2015, I started military service. And I almost ended it the same day. But I stayed, and later, when the rookie phase was over, I applied to a programming job for the rest of my service. For 6 months during March – September 2015 I was a software developer, mostly developing a single application using C++ and Qt. I also did a mobile application to easily check the food menu, using node.js and Ionic.

So yeah, that went better than I had ever hoped for. I gladly finished my service and left with plenty more C++ experience, and a job certificate.

Present day

Starting from September 2015, I am studying for a gamedev-centric software engineer’s degree in Kajaani University of Applied Sciences. This was a pretty clear choice, as I didn’t have much contacts in the software industry, so getting a job would’ve been very difficult. I have met many like-minded people who are all interested in the same things as I am, so no regrets!

And that’s about it. Wow.. I rarely look back at my previous projects like this. But anyway, my next post will probably be about learning Unreal Engine, which I have been doing recently. I will also start blogging about developing a game engine.. soon-ish.