Updating A C Extension For Ruby 1.9.1

One of my favourite C extensions for Ruby 1.8.6 is Mixology. Mixology makes it possible to dynamically mix and unmix modules from inheritance chains and can be used to do some very interesting things. Just some of the applications utilizing this kind of functionality include _why’s mixico,  the decorator pattern, and the state pattern.

Unfortunately Mixology has not yet been ported to 1.9.1. Nonetheless I’ve been wanting to try my hand at writing an extension for 1.9.1 and so I thought I’d give it a go.

The first thing to do when porting an extension from an earlier version of Ruby to Ruby 1.9.1 is to just try compiling it and examine the errors that result. I did the following in a shell:

# generate the Makefile (ensure that you use ruby19 not ruby 1.8.6 to do this)
ruby19 extconf.rb

# try to compile the extension
make

# output was:
mixology.c:31: error: ‘struct RClass’ has no member named ‘super’
mixology.c:38: error: ‘struct RClass’ has no member named ‘iv_tbl’
...
# and many more errors as above

From above, even though the compile failed it does not necessarily follow that all 1.8.6 extensions will fail to compile for 1.9.1. Indeed if the extension restricted itself to the public API (functions detailed in README.EXT) it should compile for 1.9.1 with no changes necessary.

Regarding the errors in the attempted compile above it is clear that the RClass struct has changed in 1.9.1. Here is what the RClass struct looked like in 1.8.6:

struct RClass {
    struct RBasic basic;
    struct st_table *iv_tbl;
    struct st_table *m_tbl;
    VALUE super;
};

And here is how it looks in 1.9.1:

struct RClass {
    struct RBasic basic;
    rb_classext_t *ptr;
    struct st_table *m_tbl;
    struct st_table *iv_index_tbl;
};

Note that the 1.9.1 struct also has a pointer to a rb_classext_t, so here is the struct for that:

typedef struct {
    VALUE super;
    struct st_table *iv_tbl;
} rb_classext_t;

One of the lines in mixology.c which resulted in the compilation error was this:

RCLASS(klass)->super = RCLASS(RCLASS(klass)->super)->super;

Looking at the 1.9.1 structs it is clear why we got the error we did; ‘super’ is now located in the rb_class_ext_t struct and not directly in RClass. We can rewrite the code above to compatible 1.9.1 as follows:

RCLASS(klass)->ptr->super = RCLASS(RCLASS(klass)->ptr->super)->ptr->super;

The above code compiles fine in 1.9.1 but is unwieldy to say the least. Luckily 1.9.1 comes with a bunch of handy macros that make our lives alot easier, here are a few of them:

#define RCLASS_IV_TBL(c) (RCLASS(c)->ptr->iv_tbl)
#define RCLASS_M_TBL(c) (RCLASS(c)->m_tbl)
#define RCLASS_SUPER(c) (RCLASS(c)->ptr->super)

Using the macros we can rewrite the 1.9.1 code to the following:

RCLASS_SUPER(klass) = RCLASS_SUPER(RCLASS_SUPER(klass));

Which is much easier to read.

Unfortunately 1.8.6 does not come with equivalent macros to the above so our 1.9.1 compatible code wont work for 1.8.6. This is annoying because it means we have to potentially write two versions of our extension.

However we can avoid this situation by a judicious use  of preprocessor macros and conditional compilation:

/* define 1.8.6 versions of the 1.9.1 macros if we are compiling
   for 1.8.6  */
#ifndef RUBY_19
# define RCLASS_M_TBL(c) (RCLASS(c)->m_tbl)
# define RCLASS_SUPER(c) (RCLASS(c)->super)
# define RCLASS_IV_TBL(c) (RCLASS(c)->iv_tbl)
#endif

Using the above and a few other similar constructs we can have a single mixology.c that supports both 1.8.6 and 1.9.1 Ruby versions.

Note we must define the RUBY_19 macro if we are compiling for 1.9.1; we do this in extconf.rb:

require "mkmf"

if RUBY_VERSION =~ /1.9/ then
    $CPPFLAGS += " -DRUBY_19"
end
create_makefile "mixology"

And here is the complete 1.8.6 and 1.9.1 compatible source code for mixology.c:
(click HERE for the github project)

#include "ruby.h"

/* cannot use ordinary CLASS_OF as it does not return an lvalue */
#define KLASS_OF(c) (RBASIC(c)->klass)

/* macros for backwards compatibility with 1.8 */
#ifndef RUBY_19
# define RCLASS_M_TBL(c) (RCLASS(c)->m_tbl)
# define RCLASS_SUPER(c) (RCLASS(c)->super)
# define RCLASS_IV_TBL(c) (RCLASS(c)->iv_tbl)
#endif

#ifdef RUBY_19
static VALUE
class_alloc(VALUE flags, VALUE klass)
{
    rb_classext_t *ext = ALLOC(rb_classext_t);
    NEWOBJ(obj, struct RClass);
    OBJSETUP(obj, klass, flags);
    obj->ptr = ext;
    RCLASS_IV_TBL(obj) = 0;
    RCLASS_M_TBL(obj) = 0;
    RCLASS_SUPER(obj) = 0;
    RCLASS_IV_INDEX_TBL(obj) = 0;
    return (VALUE)obj;
}
#endif

static void
remove_nested_module(VALUE klass, VALUE include_class) {

    if(KLASS_OF(RCLASS_SUPER(klass)) != KLASS_OF(RCLASS_SUPER(include_class))) {
        return;
    }
    if(RCLASS_SUPER(RCLASS_SUPER(include_class)) && BUILTIN_TYPE(RCLASS_SUPER(include_class)) == T_ICLASS) {
        remove_nested_module(RCLASS_SUPER(klass), RCLASS_SUPER(include_class));
    }
    RCLASS_SUPER(klass) = RCLASS_SUPER(RCLASS_SUPER(klass));
}

static VALUE
rb_unmix(VALUE self, VALUE module) {
    VALUE klass;

    /* check that module is valid */
    if (TYPE(module) != T_MODULE)
        rb_raise(rb_eArgError, "error: parameter must be a module");

    for (klass = KLASS_OF(self); klass != rb_class_real(klass); klass = RCLASS_SUPER(klass)) {
        VALUE super = RCLASS_SUPER(klass);
        if (BUILTIN_TYPE(super) == T_ICLASS) {
            if (KLASS_OF(super) == module) {
                if(RCLASS_SUPER(module) && BUILTIN_TYPE(RCLASS_SUPER(module)) == T_ICLASS)
                    remove_nested_module(super, module);

                RCLASS_SUPER(klass) = RCLASS_SUPER(RCLASS_SUPER(klass));
                rb_clear_cache();
            }
        }
    }
    return self;
}

static void
add_module(VALUE self, VALUE module) {
    VALUE super = RCLASS_SUPER(rb_singleton_class(self));

#ifdef RUBY_19
    VALUE klass = class_alloc(T_ICLASS, rb_cClass);
#else
    NEWOBJ(klass, struct RClass);
    OBJSETUP(klass, rb_cClass, T_ICLASS);
#endif

    if (BUILTIN_TYPE(module) == T_ICLASS) {
        module = KLASS_OF(module);
    }
    if (!RCLASS_IV_TBL(module)) {
        RCLASS_IV_TBL(module) = (void*)st_init_numtable();
    }

    RCLASS_IV_TBL(klass) = RCLASS_IV_TBL(module);
    RCLASS_M_TBL(klass) = RCLASS_M_TBL(module);
    RCLASS_SUPER(klass) = super;

    if (TYPE(module) == T_ICLASS) {
        KLASS_OF(klass) = KLASS_OF(module);
    }
    else {
        KLASS_OF(klass) = module;
    }
    OBJ_INFECT(klass, module);
    OBJ_INFECT(klass, super);

    RCLASS_SUPER(rb_singleton_class(self)) = (VALUE)klass;
}

static VALUE
rb_mixin(VALUE self, VALUE module) {

    VALUE nested_modules;
    int index;

    /* check that module is valid */
    if (TYPE(module) != T_MODULE)
        rb_raise(rb_eArgError, "error: parameter must be a module");

    rb_unmix(self, module);
    nested_modules = rb_mod_included_modules(module);

    for (index = RARRAY_LEN(nested_modules); index > 0; index--) {
        VALUE nested_module = RARRAY_PTR(nested_modules)[index - 1];
        add_module(self, nested_module);
    }

    add_module(self, module);

    rb_clear_cache();
    return self;
}

void
Init_mixology() {
    VALUE Mixology = rb_define_module("Mixology");

    rb_define_method(Mixology, "mixin", rb_mixin, 1);
    rb_define_method(Mixology, "unmix", rb_unmix, 1);
    rb_include_module(rb_cObject, Mixology);
}

7 Comments to “Updating A C Extension For Ruby 1.9.1”

  1. This is awesome. Thanks for updating Mixology to work with Ruby 1.9. I’ll pull your changes from Github this evening.

  2. Good post – I appreciate being shown what goes into this work.

    I do have a question – Would it be better to detect if the macros are undefined instead of just checking to see if RUBY_19 is defined? Ruby 20 might redefine those macros to maintain backwards compatibility and you (or someone else) will have to debug your code to figure out what the heck is going on.

  3. Why not use #ifndef RCLASS_SUPER instead, since you don’t care what version of ruby it is, but simply that the given macros exist.

  4. Aman: well technically it’s more than just the macros, the way that classes are allocated (see the class_alloc() method) matter too. But you and Matthew have a good point and it is an alternative (and perhaps better) way of doing things.

    Thanks for the comments🙂

    • great; im happy someone is taking over the project since it seems to have stalled recently.🙂 If you could be bothered adding a link to this blog somewhere in either the mixology source or documentation it would be much appreciated🙂 thanks!

  5. Quite sure. You’re now on the list of collaborators in README.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: