Blog

Model your Fame/Famix Meta-model using Graphical Editors

Mar 2, 2021

Vincent Aranega

MDE and beers lover

Building metamodels for Fame and Famix is currently done through a generator. The generator consists in various special methods that need to be implemented, eg, one that will create all metaclasses, another that will create all attributes for each metaclass,… The metamodel’s artifact are created programmatically using pure Smalltalk syntax extended with specific operators to easily express relationships and cardinalities (<>-, *-*, <>-*, …). For more details, please refer to the article about “Coaster”, written by Benoit Verhaeghe.

In this blogpost, I will show you how to use existing metamodeling diagraming tools to design your metamodel, and then to use the designed meta-model to generate your meta-model code for Fame in Moose. In another blogpost, I will show you how to use this same generator to use graphical modeling tools to generate code for Famix.

Why a Graphical Modeling Editor

The generator concept provided for Fame/Famix in Moose is practical, but can be intimidating in the first place. Moreover, discovering an existing metamodel, or modifying an existing one can be hard as it requires to switch between various methods. Some projects exist, as PlantUMLPharoGizmo, to automatically build a graphical representation of an existing metamodel. This approach considers the code of the meta-model as the main artifact and builds the diagram out of it.

In this post, we will consider the opposite: putting the diagram as the main artifact, then using the diagram as the basis for generating the code of the Fame/Famix generator. Despite its possible complexity to describe a generator for a full huge metamodel, the syntax for building the generator is easy to deal with and easy to generate. The general process is sketched in Figure 1. From the graphical modeling editor, the produced meta-model is given to a dedicated code generator that will produce the Fame/Famix Generator.

Reusing Existing Modeling Editor

Here is an interesting fact. Building a graphical modeling editor from scratch is hard. It can look easy in the first place, basically, it’s only links and boxes, but it turns out that creating an editor that allows you to create a model, enforcings rules defined by a metamodel, is a complex task (without speaking about all graphical rules that need to be injected in the editor). Consequently, we will reuse existing graphical modeling tools that already allow model metamodels.

Currently, there is no pure metamodeling editors for Pharo/Moose. The only existing project is the OpenPonk project which gives the ability to create UML models with XMI support, but UML proposes a lot of concepts that could lead to wrong design in the end (using wrong concepts). On the other end, on the MDE community, Ecore is an implementation of the EMOF and various desktop or web-based graphical editors exists. Following then the flow we described earlier, the Graphical Modeling Editor will be one of the various graphical Ecore modeling editor. The designed meta-model will be serialized in XMI (default output for those kind of editors), given to a code generator that will produce the Fame/Famix code generator. The output of the code generator will follow the tonel format so it can be easily imported by Moose/Pharo.

Aligning Ecore and Fame

As Fame, Ecore is a meta-meta-model that is used to create meta-models. Various graphical tools exist for it, so we have a little bit of choice. The entry point of our chain is an Ecore meta-model in a XMI format and the destination a Smalltalk class in a tonel format, but, the generated class will describe a generator for a Fame meta-model, not an Ecore one. Consequently, we need to align Ecore concepts with Fame concepts. This blogpost does not go into details about the full alignment between Ecore and Fame, but gives some keys that are used.

Metaclasses, EClass and Traits

Fame proposes two main concepts: FM3Class (metaclasses) and FM3Traits (traits). On the contrary, Ecore only provides the concept of EClass to represent metaclasses, which is aligned with the metaclass concept of Fame. Also, Ecore let users mark a metaclass as interface, which is a concept that comes close to the trait concept in Fame. Thus, the alignment that we will keep here is:

• EClass -> FM3Class (metaclasses)
• EClass interface -> FM3Traits (traits)

Attributes

Attributes in Fame are modeled by the concept of FM3Property. Attributes represent a property of a metaclass/trait and are usually bound to objects outside of the Fame “ecosystem” (as String that comes from Pharo for example). The same concept can be found in Ecore with the EAttribute concept. On the contrary of attributes in Fame, EAttribute in Ecore can own default values (and a set of other attributes that we will not consider here). Consequently, we can directly align simple EAttribute to Fame attribute, ignoring the default value if one is set. In this condition, the alignment is:

• EAttribute -> FM3Property

References

References in Fame are also modeled by the concept of FM3Property. They can own a set of properties and can be marked as “containement”, and they own a cardinality. References represent relationships between metaclasses. In Ecore, the EReference concept is the equivalent to FM3Property, that’s then the one we use to model relationships. The alignment is:

• EReferences -> FM3Property

Untackled Concepts from Ecore

Ecore proposes more concepts that are not directly aligned with Fame, e.g.: EEnumeration, EGenericType,… Thus, those concepts are not handled by the generator.

Designing a Meta-model for Fame

Now that we know how we will represent Fame concepts with Ecore, we can use a graphical modeling tool to design Ecore meta-model to design Fame metamodels.

For this blogpost, three tools have been tested, two web-based one and a desktop one:

• GenMyModel (web-based)
• Ecore Diagram (desktop)
• EMF.cloud (web-based)

They all work well and have their pros and cons. As example/illustration of this section I used GenMyModel (I worked there for some years, I know well the tool), but the design process is the same whatever the used tool.

When you design your meta-model for Fame using a graphical Ecore editor, you just focus on the concepts you want to represent. The meta-model we are going to design is the one presented by Benoit in his article about “Coaster”.

The designed Ecore meta-model can be found at this address and it looks like this:

In addition, we also set two specific properties of the base EPackage: nsURI and nsPrefix. While the nsURI is more internal/maintenance related (it can be used to propose a unique identifier towards this version of the metamodel), nsPrefix is really important as it will set the prefix that will be used later for the generation of the generator. I thus set the two values to:

• nsURI = https://coaster/1.0
• nsPrefix = Ct

With this, we are ready to generate the code of our generator.

Generate your Generator

The code generator is coded in Python using the PyEcore library to be able to decode/resolve Ecore files. The generator goes across the full meta-model at least once to perform model validation, tries to correct some issues (like spaces at the beginning or end or names), and raises an error if it cannot generate the code for some reasons. Once the meta-model is validated, it generates the Fame generator code using Mako, a template engine.

Install the code generator

The generator installation is pretty simple if you have the right tools. You can find the code on GitHub at this address. You can install all the dependencies manually on your system, but the proper way of dealing with the dependencies would be to use a virtual environment (a virtualenv). Virtualenvs creation and activation can be somehow complicated if you are not used to it. To ease this step, the repository contains a Pipfile to be used with pipenv. If you don’t have pipenv installed on your machine, feel free to install it, it helps you creating virtualenvs from Pipfile with all the required dependencies and ease also their activation. In our case, with pipenv, the installation is pretty forward:

$ git clone https://github.com/aranega/famegenerator.git
$ cd famegenerator
$ pipenv install

And you’re good to go, pipenv creates a virtualenv and installs all the dependencies isolated in your virtualenv. Now, each time you want to use the created virtualenv, you can just enter in the directory where the code of the generator is installed and just activate the virtualenv:

$ pipenv shell

Generate the code of your Generator

If you did your meta-model online, there are two options for you. Either you can download the .ecore on your machine and generate the code from there, or you can give to the generator a valid URL where the XMI of your ecore meta-model is located. Whatever option you’ll choose, you can generate the code of your generator this way:

$ python famegen.py <URL_OR_FILE> -o mymeta.class.st

This will generate the code of your generator in a tonel format in the file mymeta.class.st. If you don’t use the option -o, the generated code is displayed on standard output.

Here is the line used to generate the generator of the Coaster meta-model:

$ python famegen.py https://app.genmymodel.com/api/projects/_vgiYcHuVEeuCM8KqVoRWiA/xmi -o /tmp/coaster.class.st

Here is the content of the file:

Class {
  #name : #CoasterGenerator,
  #superclass : #FamixMetamodelGenerator,
  #instVars : [
    'shape',
    'coaster',
    'creator',
    'country',
    'round',
    'square',
    'oval',
    'brewery'
  ],
  #category : #Coaster
}


{ #category : #accessing }
CoasterGenerator class >> packageName [
  ^ 'Coaster'
]

{ #category : #accessing }
CoasterGenerator class >> prefix [
  ^ 'Ct'
]


{ #category : #definition }
CoasterGenerator >> defineClasses [
  super defineClasses.

  shape := builder newClassNamed: #Shape.
  coaster := builder newClassNamed: #Coaster.
  creator := builder newClassNamed: #Creator.
  country := builder newClassNamed: #Country.
  round := builder newClassNamed: #Round.
  square := builder newClassNamed: #Square.
  oval := builder newClassNamed: #Oval.
  brewery := builder newClassNamed: #Brewery.
]

{ #category : #definition }
CoasterGenerator >> defineHierarchy [
  super defineHierarchy.

  round --|> shape.
  square --|> shape.
  oval --|> shape.
  brewery --|> creator.
]

{ #category : #definition }
CoasterGenerator >> defineProperties [
  super defineProperties.

  coaster property: #image type: #String.
  coaster property: #number type: #Number.
  coaster property: #owned type: #Boolean.
  creator property: #name type: #String.
  country property: #name type: #String.
]

{ #category : #definition }
CoasterGenerator >> defineRelations [
  super defineRelations.

   (shape property: #coasters)  *-* (coaster property: #shapes).
   (coaster property: #country)  *-<> (country property: #coasters).
   (coaster property: #creator)  *-<> (creator property: #coasters).
]

We find back all the information we put in our design, relationships, names, metaclasses, … This file can be directly imported in Moose and used to generate the meta-model code:

(TonelParser parseString: ('/tmp/coaster.class.st' asFileReference contents )) do: #load. "Load the generated generator"
CoasterGenerator generate. "Generate the meta-model code"

And Next Time…

That’s all folks for this post. Next time, we will discuss about how to generate a dedicated meta-model for Famix instead of only Fame. The exact same code generator will be used, but this time, we will have to deal with the desktop Ecore diagram tool for technical reasons.

Happy metamodeling!

Tags:

• infrastructure

Coasters collection

Feb 15, 2021 - Last update: Mar 4, 2021

Benoit Verhaeghe

Moose expert

Introduction

I’m a coasters collector. I’m not a huge collector but I want to inventory them in one place. For sure, I can create a PostgreSQL database. But, at the same time, it appears that I can also design my collection using Moose.

So, you’re going to use a complete system analysis software to manage your coasters collection?

Exactly! And why? Because I think it’s simpler.

Design

As for every software system, the first step is to design the model. In my case, I want to represent a collection of coasters. Let’s say a coaster is an entity. It can belong to a brewery or not (for example event coasters). A coaster also has a form. It can be round, squared, oval, or others. A Coaster can also be specific to a country. Because it is a collection, I can register coaster I own and other I do not. Finally, each coaster can have an associated image.

From this description of the problem, I designed my UML schema:

The most complicated part is done. We just need to implement the meta-model in Moose now 😄.

Implementation

First of all, we’ll need a Moose 8 image. You can find everything you need to install Moose in the moose-wiki.

Configuration

Ok! Let’s create a generator that will generate for us the meta-model. We only need to describe the meta-model in the generator. We will name this generator CoasterCollectorMetamodelGenerator.

FamixMetamodelGenerator subclass: #CoasterCollectorMetamodelGenerator
    slots: {  }
    classVariables: {  }
    package: 'CoasterCollector-Model-Generator'

The generator needs to define two methods class side for the configuration:

• #packageName defines where the meta-model will be generated
• #prefix defines the prefix of each class when they are generated.

We used for #packageName:

CoasterCollectorMetamodelGenerator class >>#packageName
    ^ #'CoasterCollector-Model'

We used for #prefix:

CoasterCollectorMetamodelGenerator class >>#prefix
    ^ #'CC'

Now, we have to define the entities, their properties, and their relations.

Define entities

A meta-model is composed of entities. In our case, it corresponds to the entities identified in the UML. We use the method #defineClasses to define the entities of our meta-model.

CoasterCollectorMetamodelGenerator>>#defineClasses
    super defineClasses.
    coaster := builder newClassNamed: #Coaster.

    country := builder newClassNamed: #Country.

    shape := builder newClassNamed: #Shape.
    round := builder newClassNamed: #Round.
    square := builder newClassNamed: #Square.
    oval := builder newClassNamed: #Oval.

    creator := builder newClassNamed: #Creator.
    brewery := builder newClassNamed: #Brewery

We also need to define the hierarchy of those entities:

CoasterCollectorMetamodelGenerator>>#defineHierarchy
    super defineHierarchy.
    brewery --|> creator.
    oval --|> shape.
    square --|> shape.
    round --|> shape

Define properties

As we have defined the classes, we defined the properties of the entities using the #defineProperties method.

defineProperties
    super defineProperties.
    creator property: #name type: #String.
    country property: #name type: #String.
    coaster property: #image type: #String.
    coaster property: #owned type: #Boolean

In this example, we did not use Trait already created in Moose. However, it is possible to use the Trait TNamedEntity to define that countries and creators have a name instead of using properties.

Define relations

Finally, we defined the relations between our entities:

defineRelations
    super defineRelations.
    (coaster property: #shape) *- (shape property: #coasters).
    (coaster property: #country) *- (country property: #coasters).
    (coaster property: #creator) *- (creator property: #coasters)

Generate

Once everything is defined, we only need to use the generator to build our meta-model.

CoasterCollectorMetamodelGenerator generate

The generation creates a new package with our entities. It also generates a class named Model used to create an instance of our meta-model.

Create my collection

I have created my meta-model. Now I need to fill my collection. First of all, I will create a collection of coasters. To do so, I instantiate a model with: model := CCModel new. And now I can add the entities of my real collection in my model and I can explore it in Moose.

For example, to add a new brewery I execute: model add: (CCBrewery new name: 'Badetitou'; yourself).

The code is available on github.

Save the collection

Once I have created the collection, I can save it using the Moose export format (currently JSON and mse). To do so, I execute the following snippet:

'/my/collection/model.json' asFileReference ensureCreateFile
    writeStreamDo: [ :stream | model exportToJSONStream: stream ]

Then I can select where I want to export my model.

To import it back into an image, I use the following code

'/my/collection/model.json' asFileReference
    readStreamDo: [ :stream | model := CCModel importFromJSONStream: stream ]

Tags:

• meta-model

Micro-Visitors for Parsing Programming Languages

Jan 26, 2021

Nicolas Anquetil

Moose expert

For Moose, I had to design a number of parsers for various languages (Java, Ada, C/C++, PowerBuilder). If you have already done that, you will know that the Visitor pattern is a faithful ally. To help me in this, I came with the concept of “micro visitor” allowing to modularize visitors.

Parsing and Visitors

Parsing source code starts with a grammar of the programming language and an actual parser that creates an Abstract syntax Tree (AST) of the program.

For many programming languages, the AST can contain tens of different nodes. The way to master this complexity is to use visitors. A visitor is a class with one method (traditionaly visitXYZ(XYZ node)) for each possible type of node in the AST. Each method treats the current node and delegates to other methods treating the nodes below it.

For a parser like VerveineJ (Java to MSE importer) the visitor class reached 2000 lines of code and became difficult to maintain as there are also interactions between visiting methods because the treatment of a node down in the AST may depend on what are its parent nodes. For example, in Java, ThisExpression node may be found in different situations:

• Return the instance running the current method: this.attribute
• Return the enclosing object of the current instance: AClass.this
• Invoke current class constructor: this(...)

Therefore the treatment in visitThisExpression( ThisExpression node) may depend on which method called it. This makes it more complex to develop and maintain all the “visitXYZ” methods.

Micro-Visitor

On the other hand, a visitor typically has a small state:

• the name of the file being parsed;
• a context stack of the visit (eg visiting a method, inside a class, inside a file);
• a model of the result being built by the visitor (eg a Moose model).

As a result, I came up with the notion of micro-visitors specialized for a single task. For example, for VerveineJ, I have 10 (concrete) micro-visitors, 4 to create entities and 6 to create dependencies between them:

• VisitorPackageDef, creating Famix packages;
• VisitorClassMethodDef, creating Famix classes and methods;
• VisitorVarsDef, creating Famix attribute, parameter, local variable definition;
• VisitorComments, creating comments in all Famix entities;
• VisitorInheritanceRef, creating inheritances between classes
• VisitorTypeRefRef, creating reference to declared types;
• VisitorAccessRef, creating accesses to variables;
• VisitorInvocRef, creating invocation dependencies between methods;
• VisitorAnnotationRef, handling annotations on entities;
• VisitorExceptionRef, handling declared/catched/thrown exceptions.

The resulting visitors are much smaller (around 600 lines of code for the three more complex: VisitorInvocRef, VisitorClassMethodDef, VisitorAccessRef ; less than 150 lines of code for VisitorPackageDef and VisitorExceptionRef) and thus easier to define and maintain. Also, because the visitor is specialized, there are less dependencies between the methods: VisitorInvocRef only treats ThisExpression when it is a constructor invocation.

Using Micro-Visitors

The overhead on the execution is small as each visitor is specialized and does not need to go through all the AST (eg a visitor for function declaration in C would not have to visit the body of these functions since they cannot contain other function declarations).

Micro-visitors can be used independantly one of the other (in sequence) as in VerveineJ where each visitor is called one after the other (by the FamixRequestor class) to visit the full AST. The “orchestrator” object owns the state and pass it to each visitor in turn.

Micro-visitors can also call one another (in delegation). For example for PowerBuilder, there is one main visitor (PowerBuilder-Parser-Visitor.PWBCompilationUnitVisitor, visiting the AST for a source file) and 7 (concrete) micro-visitors:

• PWBTypeDeclarationVisitor, visiting type declarations;
• PWBBehaviouralDeclarationVisitor, visiting function/routine definitions;
• PWBVariableDeclarationVisitor, visiting declarations of all kind of variables;
• PWBTypeReferenceToIdentifierVisitor, visiting references to type names (for example in variable declarations);
• PWBStatementsVisitor, visiting statements in the body of behaviourals;
• PWBExpressionsVisitor, visiting expressions in statements;
• PWBBehaviouralInvocationVisitor, visiting the invocation of behavioural in expressions.

In this case, the main visitor (PWBCompilationUnitVisitor) owns the state and its auxiliary visitors get this state from their respective parent visitor:

• PWBCompilationUnitVisitor spawns a PWBBehaviouralDeclarationVisitor when it encounters a function definition, this one spawns a PWBStatementsVisitor to visit the body of the function, PWBStatementsVisitor spawns a PWBExpressionsVisitor to visit expressions found in the statements.
• if the PWBExpressionsVisitor needs to access the context stack, it asks to its parent PWBStatementsVisitor, that asks to its parent PWBBehaviouralDeclarationVisitor, that asks to the PWBCompilationUnitVisitor.

Tags:

• infrastructure

Newer posts