Famix-tools

9 posts with the tag “Famix-tools”

Mixing up analysis of Project Architecture (Famix) and its history (Mining Software Repository)

Aug 27, 2026

Famix is the meta-model family of Moose allowing one to analyse the dependencies of a project and its global architecture. But sometimes, you want more, you want to see what are the part of the code that are modified most, are critical, etc. It is for instance what people do when using Sonar, Coverage analysis, and so on.

To explore the history of existing projects, Mining Software Repositories (MSR) is the way to go. Moose also includes a set of tools to explore the history of a project, and to combine it with the Famix analysis. The project is named GitProjectHealth, and the full documentation is available here.

But let’s today perform an example. I have this nice HR application that I want to investigate. It is large, and I want to retrieve information between the architecture and the history of the project.

So let’s do a step by step blog about this story.

Creating the Famix model

I am working with Moose12. But this story should last for future version (I mean, I don’t know how long, but it should not change a lot) First, I clone the project, and then, I install its dependency with maven mvn install.

Then, I build a nice model using the docker version of VerveineJ.

docker run -e JAVA_XMX=-Xmx16G -v /path/to/hr:/src -v /path/to/.m2/repository:/dependency ghcr.io/evref-bl/verveinej:v4.2.6 -anchor assoc -alllocals -format json

I use the version 4.2.6 of VerveineJ, because I am with Moose12, but don’t hesitate to use the latest version of VerveineJ if you are with a more recent version of Moose.

Then, I load the model in Moose, and I can start to explore the architecture of the project. To load the model, I simply drag and drop the json file in Moose, and then I can start to explore the model.

Retrieving the history of the project

To retrieve the history of the project, I use the GitProjectHealth tool. This project is not included in the Moose distribution by default, but you can download it by executing the following command in a Moose playground:

Metacello new
  repository: 'github://moosetechnology/GitProjectHealth:main/src';
  baseline: 'GitLabHealth';
  onConflict: [ :ex | ex useIncoming ];
  onUpgrade: [ :ex | ex useIncoming ];
  onDowngrade: [ :ex | ex useLoaded ];
  load

Then, because it is a large HR application, I will perform the retrieving of the history locally using git.

rhModel := GLHModel new.
repository := GLHRepository new
  cacheAt: #localImporterReference
  put: '/path/to/hr' asFileReference;
  yourself.
project := GLHProject new.
project repository: repository.

rhModel add: repository.
rhModel add: project.

localImporter := GitLocalModelImporter new.
localImporter withFiles: true.
localImporter withCommitDiffs: true.
localImporter withDiffRanges: false.
localImporter glhModel: rhModel.
localImporter withCommitsSince: 30 years.
localImporter importRepository: repository.

rhModel flush.
rhModel entities do: #flush.

You’ll notice several things in the code above.

I created a fake project and repository to store the history of the project. This is to help the local importer and might be improved.
I use the withFiles option to retrieve the files of the project, and the withCommitDiffs option to retrieve the diffs of the commits. The withDiffRanges option is set to false because I don’t need it for now.
I use the withCommitsSince: 30 years option to retrieve all the commits of the project. You can change this value to retrieve only the last commits if you want.
I use the flush method to remove cache that is computed by the local importer, this is for storage efficiency.

Once it is done, I have the full history of the project, but not connected to the exsiting model of the code.

Connecting the models

We want now to connect the GitProjectHealth model, with the Famix model. There is a tool dedicated for this usage in GitProjectHealth, it is called GPCGitToFamixConnector.

The project links the types of the Famix model with the files retrieved by the GitProjectHealth model based on the files path. To use it, simply execute the following code:

GPCGitToFamixConnector new
  famixModel: myFamixModel;
  glhProject: project;
  connect.

it is possible that GPCGitToFamixConnector is not loaded in your Moose image. It is is the case, load the missing package from Iceberg, or check in the baseline of GitProjectHealth if it is included in the version you are using, there should be a group named “Famix”

And, that’s it… yeah, it is quite easy to connect the two models, and now you can explore the history of the project in the context of its architecture.

You want to search for the types the most modified? One classic Moose query will do the job:

(myFamixModel allUsing: FamixTType) sorted: [ :type | type appliedDiffs size ] descending

Improving the generation of visitors

May 20, 2026

Cyril Ferlicot-Delbecque

Research engineer at Inria

Context

Recently, I needed to implement a feature in FAST-Python requiring a visitor for my metamodel. People knowing me knows that I if I can automatize a task to save time and to keep the code coherant with the model, I’ll go for it. Nicolas Anquetil and Clotilde Toullec recently started to implement a visitor generator, so I tried it. Since it was a POC, I encountered some problems.

I’ll explain in this blog post how to use this visitor generator, what problems I encountered and the solutions I proposed.

Using the visitor generator

Once my changes will be integrated, using the visitor generator is as simple as adding one method on the class side of your generator. For example, in FAST-Python:

FASTPythonMetamodelGenerator class>>metamodelToolGenerators

  ^ { FamixVisitorGenerator }

This will tell the generator to also generate a visitor trait.

It is possible to also customize the package in which the visitor is generated by overridin #packageNameForVisitor:

FASTPythonMetamodelGenerator class>>packageNameForVisitor

  ^ #'FAST-Python-Visitor'

Problems I faced

While using this visitor generator I faced a few problems.

Dependency on sub metamodels

Problem

The first one I encountered is that FAST-Python is depending on FAST that is depending on Famix that is depending on MooseQuery.

In order to have the FAST-Python visitor working, all dependencies needed to have a visitor themselves which was not the case. So I had to generate a FAST visitor for example. I could do it since I am part of the moosetechnology organization, but this is a hassle.

Solution

The solution I found was to generate standalone visitors. Instead of using the visitors of the sub metamodels, now we generate a visitor containing the visit of all entities of the model and the visit of the remote trait used in the model.

This can seems like a lot of useless code at first, but it has two advantages:

Visitors and now standalone
This allow to customize the visit of remote traits for the language we are currently working on. This will be important for one of the other problems I encountered.

Infinit loops in visitors

Problem

The second problem I faced was that the generated visitor had infinit loops while I tried to use it. The reason is simple, when visiting an entity, we were visiting all its relations, but Moose is a cyclic graph since we have relations to the contained entities and relations to the containers of the entity.

Solution

Since most analysis that we are doing are “top down” (meaning that we start by the top level entities of our project and we visit their children) and we do not often need a “bottom up” visitor (for this we can just iterate on parents), I decided to exclude the visit of parent entities from the visit.

Double visit of some relations

Problem

In some cases I noticed the we visited two times the same nodes. This is due to the fact that Famix model mix the usage of superclasses and traits. Here is a simple case to understant what is happening:

Double visit example

Here we see that a FASTPyReturnStatement inherits from FASTPyStatement. This class is using FASTTStatement. FASTPyReturnStatement also uses the trait FASTPyReturnStatement, but this trait also uses FASTTStatement.

Now when we visit FASTPyReturnStatement, we end up visiting two times FASTTStatement. Once via its superclass and once via its trait composition.

Solution

The solution I proposed is that when we visit a trait composition, we check the users of the trait and if some of them already visit the trait in their superclasses, we skip the visit of this relation.

With this, #visitFASTPyReturnStatement in the context of FAST-Python become:

visitFASTTReturnStatement: aTReturnStatement

  <generated>
  "We should visit FASTTStatement but all its users in this language already visit it in their superclasses so we skip the call here.".

  self visitEntity: aTReturnStatement expression

In some cases, we need to skip the visit for some entities, but not all of them. For example, with conditional statements. In that case we generate a method like this:

visitFASTTConditionalStatement: aTConditionalStatement

  <generated>
  "We do not visit all behaviorals because some classes already visit it in their superclasses in this language implementation. Visiting them here also would cause a double visit of this trait."
  ({ FASTPyIfStatement . FASTPyWhileStatement } includes: aTConditionalStatement class)
    ifFalse: [ self visitFASTTStatement: aTConditionalStatement ].

  self visitFASTTWithCondition: aTConditionalStatement

This is possible only because we generate standalone visitors now.

Conclusion

With those changes I’m hoping it will be easier to generate and use visitors with Famix. The big advantage is that if we do not need to touch the visitor by hand, it will follow all the evolutions of the metamodel.

Have fun with this :)

First look at GitProjectHealth

Mar 24, 2025

Benoit Verhaeghe

Moose expert

When it comes to understand a software system, we are often focusing on the software artifact itself. What are the classes? How they are connected with each other?

In addition to this analysis of the system, it can be interesting to explore how the system evolves through time. To do so, we can exploit its git history. In Moose, we developed the project GitProjectHealth that enables the analysis of git history for projects hosted by GitHub, GitLab, or BitBucket. The project also comes with a set of metrics one could use directly.

First use of GitProjectHealth

GitProjectHealth is available in the last version of Moose, it can be easily installed using a Metacello script in a playground.

Metacello new
  repository: 'github://moosetechnology/GitProjectHealth:main/src';
  baseline: 'GitLabHealth';
  onConflict: [ :ex | ex useIncoming ];
  onUpgrade: [ :ex | ex useIncoming ];
  onDowngrade: [ :ex | ex useLoaded ];
  load

Example on Famix

For this first blog post, we will experiment GitProjectHealth on the Famix project. Since this project is a GitHub project, we first create a GitHub token that will give GitProjectHealth the necessary authorization.

Then, we import the moosetechnology group (that hosts the Famix project).

glhModel := GLHModel new.

githubImporter := GithubModelImporter new
  glhModel: glhModel;
  privateToken: '<private token>';
  yourself.

githubImporter withCommitsSince: (Date today - 100 days).
group := githubImporter importGroup: 'moosetechnology'.

This first step allows us to get first information on projects. For instance, by inspecting the group, we can select the “Group quality” view and see the group projects and the last status of their pipelines.

Group Quality view for moosetechnology

Then, by navigating to the Famix project and its repository, you can view the Commits History.

alt text .

It is also possible to explore the recent commit distribution by date and author

commit distribution .

In this visualization, we discover that the most recent contributors are “Clotilde Toullec” and “CyrilFerlicot”. The “nil” refers to a commit authors that did not fill GitHub with their email. It is anquetil (probably the same person as “Nicolas Anquetil”). The square without name is probably someone that did not fill correctly the local git config for username.

Who modified my code? - code churn

A popular metric when looking at git history is the code churn. Code churn refer to edit of code introduced in the past. It corresponds to the percentage of code introduced in a commit and then modified in other comments during a time period (e.g in the next week). However many code churn definitions exit.

The first step is thus to discover what commits modified my code. To do so, we implemented in GitProjectHealth information about diff in commit.

To extract this information, we first ask GitProjectHealth to extract more information for the commits of the famix project.

famix := group projects detect: [ :project | project name = 'Famix' ].

"I want to go deeper in analysis for famix repository, so I complete commit import of this project"
githubImporter withCommitDiffs: true.
famix repository commits do: [ :commit | githubImporter completeImportedCommit: commit ].

Then, when inspecting a commit, it is possible to switch to the “Commits tree” view.

Commit Tree

Here how to read to above example

The orange square “Remove TClassWithVisibility…” is the inspected commit.
The gray square is the parent commit of the selected ones.
The red squares are subsequent commits that modify at least one file in common with the inspected commit
The green squares are commits that modifies other part of the code

Based on this example, we see that Clotilde Toullec modifies code introduced in selected commits in three next commits. Two are Merged Pull Request. This can represent linked work or at least actions on the same module of the application.

Can we go deeper in the analysis?

Connect GitProjectHealth

It is possible to go even deeper in the analysis by connecting GitProjectHealth with other analysis. This is possible by connecting metamodels. For instance, it is possible to link GitProjectHealth with Jira system, of Famix models. You can look at the first general documentation, or stay tune for the next blog post about GitProjectHealth!

Control Flow Graph for FAST Fortran

Mar 8, 2025

Nicolas Anquetil

Moose expert

A Control Flow Graph analysis for FAST Fortran

Control Flow Graphs (CFG) are a common tool for static analyzis of a computation unit (eg. a method) and find some errors (unreachable code, infinite loops)

It is based on the concept of Basic Block: a sequence of consecutive statements in which flow of control can only enter at the beginning and leave at the end. Only the last statement of a basic block can be a branch statement and only the first statement of a basic block can be a target of a branch.

There are two distinctive basic blocks:

Start Block: The entry block allows the control to enter into the control flow graph. There should be only one start block.
Final Block: Control flow leaves through the exit block. There may be several final blocks.

The package FAST-Fortran-Analyses in https://github.com/moosetechnology/FAST-Fortran contains classes to build a CFG of a Fortran program unit (a main program, a function, or a subroutine).

Creating the FAST Model

We must first create a FAST model of a Fortran program. For this we need an external parser. We currently use fortran-src-extras from https://github.com/camfort/fortran-src-extras.

To run it on a fortran file you do:

fortran-src-extras serialize -t json -v77l encode <fortran-file.f>

This will produce a json AST of the program that we can turn into a FAST-Fortran AST.

If you have fortran-src-extras installed on your computer, all this is automated in FAST-Fortran

 <fortran-file.f> asFileReference
  readStreamDo: [ :st |
    FortranProjectImporter new getFASTFor: st contents ]

This script will create an array of ASTs from the <fortran-file.f> given fortran file. If there are several program units in the file, there will be several FAST models in this array. In the example below, there is only one program, so the list contains only the AST for this program.

We will use the following Fortran-77 code:

      PROGRAM EUCLID
*     Find greatest common divisor using the Euclidean algorithm
        PRINT *, 'A?'
        READ *, NA
        IF (NA.LE.0) THEN
          PRINT *, 'A must be a positive integer.'
          STOP
        END IF
        PRINT *, 'B?'
        READ *, NB
        IF (NB.LE.0) THEN
          PRINT *, 'B must be a positive integer.'
          STOP
        END IF
        IA = NA
        IB = NB
    1   IF (IB.NE.0) THEN
          ITEMP = IA
          IA = IB
          IB = MOD(ITEMP, IB)
          GOTO 1
        END IF
        PRINT *, 'The GCD of', NA, ' and', NB, ' is', IA, '.'
        STOP
        END

Creating the CFG

From the FAST model above, we will now create a Control-Flow-Graph:

 <FAST-model> accept: FASTFortranCFGVisitor new

The class FASTFortranCFGVisitor implements an algorithm to compute basic blocks from https://en.wikipedia.org/wiki/Basic_block.

This visitor goes throught the FAST model and creates a list of basic blocks that can be inspected with the #basicBlocks method.

There is a small hierarchy of basic block classes:

FASTFortranAbstractBasicBlock, the root of the hierarchy. It contains #statements (which are FAST statement nodes). It has methods to test its nature: isStart, isFinal, isConditional. It defines an abstract method #nextBlocks that returns a list of basic blocks that this one can directly reach. Typically there are 1 or 2 next blocks, but Fortran can have more due to “arithmetic IF”, “computed GOTO” and “assigned GOTO” statements.
FASTFortranBasicBlock, a common basic block with no branch statement. If it is final, its #nextBlocks is empty, otherwise it’s a list of 1 block.
FASTFortranConditionalBasicBlock, a conditional basic block. It may reach several #nextBlocks, each one associated with a value, for example true and false. The method #nextBlockForValue: returns the next block associated to a given value. In our version of CFG, a conditional block may only have one statement (a conditional statement).

You may have noticed that our blocks are a bit different from the definition given at the beginning of the blog-post:

our “common” blocs cannot have several next, they never end with a conditional statement;
our conditional blocks can have only one statement.

For the program above, the CFG has 10 blocks.

the first block is a common block and contains 2 statements, the PRINT and the READ;
its next bloc is a conditional block for the IF. It has 2 next blocs:
- true leads to a common block with 2 statements, the PRINT and the STOP. This is a final block (STOP ends the program);
- false leads to the common block after the IF
…

Visualize the CFG

As a first analysis tool, we can visualize the CFG. Inspecting the result of the next script will open a Roassal visualization on the CFG contained in the FASTFortranCFGVisitor.

FASTFortranCFGVisualization on: <aFASTFortranCFGVisitor>

For the program above, this gives the visualization below.

the dark dot is the starting block (note that it is a block and contains statements);
the hollow dots are final blocks;
it’s not the case here, but a block may also be start and final (if there are no conditional blocks in the program) and this would be represented by a “target”, a circle with a dot inside;
a grey square is a comon block;
a blue square is a conditional block;
hovering the mouse on a block will bring a pop up with the list of its statements (this relies on the FASTFortranExporterVisitor)

"Viualizing the Control Flow Graph"

One can see that:

the start block has 2 associated statements (PRINT and READ);
there are several final blocks, due to the STOP statements;
there is a loop at the bottom left of the graph where the last blue conditional block is “IF (IB.NE.0)” and the last statement of the grey block (true value of the IF), is a GOTO.

Other analyses

There are little analyses for now on the CFG, but FASTFortranCFGChecker will compute a list of unreachableBlocks that would represent dead code.

Control flow graphs may also be used to do more advanced analyses and possibly refactor code. For example, we mentioned the loop at the end of our program implemented with a IF statement and a GOTO. This could be refactored into a real WHILE loop that would be easier to read.

This is left as an exercise for the interested people 😉

Adapting to other languages

Building a control flow graph is language dependant to identify the conditional statements, where they lead, and the final statements.

But much could be done in FAST core based on FASTTReturnStatement and a (not yet existing at the time of writing) FASTTConditionalStatement.

Inspiration could be taken from FASTFortranCFGVisitor and the process is not overly complicated. It would probably be even easier for modern languages that do not have the various GOTO statements of Fortran.

Once the CFG is computed, the other tools (eg. the visualization) should be completely independant of the language.

All hands on deck!

Some tools on FAST models

Mar 7, 2025

Nicolas Anquetil

Moose expert

The package FAST-Core-Tools in repository https://github.com/moosetechnology/FAST offers some tools or algorithms that are running on FAST models.

These tools may be usable directly on a specific language FAST meta-model, or might require some adjustements by subtyping them. They are not out-of-the-shelf ready to use stuff, but they can provide good inspiration for whatever you need to do.

Dumping AST

Writing test for FAST can be pretty tedious because you have to build a FAST model in the test corresponding to your need. It often has a lot of nodes that you need to create in the right order with the right properties.

This is where FASTDumpVisitor can help by visiting an existing AST and “dump” it as a string. The goal is that executing this string in Pharo should recreate exactly the same AST.

Dumping an AST can also be useful to debug an AST and checking that it has the right properties.

To use it, you can just call FASTDumpVisitor visit: <yourAST> and print the result. For example:

FASTDumpVisitor visit:
  (FASTJavaUnaryExpression new
    operator: '-' ;
    expression:
      (FASTJavaIntegerLiteral new
        primitiveValue: '5'))

will return the string: FASTJavaUnaryExpression new expression:(FASTJavaIntegerLiteral new primitiveValue:'5');operator:'-' which, if evaluated, in Pharo will recreate the same AST as the original.

Note: Because FAST models are actually Famix models (Famix-AST), the tools works also for Famix models. But Famix entities typically have more properties and the result is not so nice:

FASTDumpVisitor visit:
  (FamixJavaMethod new
    name: 'toto' ;
    parameters: {
      FamixJavaParameter new name: 'x' .
      FamixJavaParameter new name: 'y'} ).

will return the string: FamixJavaMethod new parameters:{FamixJavaParameter new name:'x';isFinal:false;numberOfLinesOfCode:0;isStub:false.FamixJavaParameter new name:'y';isFinal:false;numberOfLinesOfCode:0;isStub:false};isStub:false;isClassSide:false;isFinal:false;numberOfLinesOfCode:-1;isSynchronized:false;numberOfConditionals:-1;isAbstract:false;cyclomaticComplexity:-1;name:'toto'.

Local symbol resolution in an AST

By definition an AST (Abstract Syntax Tree) is a tree (!). So the same variable can appear several time in an AST in different nodes (for example if the same variable is accessed several times).

The idea of the class FASTLocalResolverVisitor is to relate all uses of a symbol in the AST to the node where the symbol is defined. This is mostly useful for parameters and local variables inside a method, because the local resover only looks at the AST itself and we do not build ASTs for entire systems.

This local resolver will look at identifier appearing in an AST and try to link them all together when they correspond to the same entity. There is no complex computation in it. It just looks at names defined or used in the AST.

This is dependant on the programming language because the nodes using or defining a variable are not the same in all languages. For Java, there is FASTJavaLocalResolverVisitor, and for Fortran FASTFortranLocalResolverVisitor.

The tool brings an extra level of detail by managing scopes, so that if the same variable name is defined in different loops (for example), then each use of the name will be related to the correct definition.

The resolution process creates:

In declaration nodes (eg. FASTJavaVariableDeclarator or FASTJavaParameter),a property #localUses will list all referencing nodes for this variable;
In accessing nodes, (eg. FASTJavaVariableExpression), a property #localDeclarations will lists the declaration node corresponding this variable.
If the declaration node was not found a FASTNonLocalDeclaration is used as the declaration node.

Note: That this looks a bit like what Carrefour does (see /blog/2022-06-30-carrefour), because both will bind several FAST nodes to the same entity. But the process is very different:

Carrefour will bind a FAST node to a corresponding Famix node;
The local resolver binds FAST nodes together.

So Carrefour is not local, it look in the entire Famix model to find the entity that matches a FAST node. In Famix, there is only one Famix entity for one software entity and it “knows” all its uses (a FamixVariable has a list of FamixAccess-es). Each FAST declaration node will be related to the Famix entity (the FamixVariable) and the FAST use nodes will be related to the FamixAccess-es.

On the other hand, the local resolver is a much lighter tool. It only needs a FAST model to work on and will only bind FAST nodes between themselves in that FAST model.

Round-trip validation

For round-trip re-engineering, we need to import a program in a model, modify the model, and re-export it as a (modified) program. A lot can go wrong or be fogotten in all these steps and they are not trivial to validate.

First, unless much extra information is added to the AST, the re-export will not be syntactically equivalent: there are formatting issues, indentation, white spaces, blank lines, comments that could make the re-exported program very different (apparently) from the original one.

The class FASTDifferentialValidator helps checking that the round-trip implementation works well. It focuses on the meaning of the program independently of the formatting issues. The process is the follwing:

parse a set of (representative) programs
model them in FAST
re-export the programs
re-import the new programs, and
re-create a new model

Hopefully, the two models (2nd and last steps) should be equivalent This is what this tool checks.

Obviously the validation can easily be circumvented. Trivially, if we create an empty model the 1st time, re-export anything, and create an empty model the second time, then the 2 models are equivalent, yet we did not accomplish anything. This tool is an help for developers to pinpoint small mistakes in the process.

Note that even in the best of conditions, there can still be subtle differences between two equivalent ASTs. For example the AST for “a + b + c” will often differ from that of “a + (b + c)”.

The validator is intended to run on a set of source files and check that they are all parsed and re-exported correctly. It will report differences and will allow to fine tune the comparison or ignore some differences.

It goes through all the files in a directory and uses an importer, an exporter, and a comparator. The importer generates a FAST model from some source code (eg. JavaSmaCCProgramNodeImporterVisitor); the exporter generates source code from a model (eg. FASTJavaExportVisitor); the comparator is a companion class to the DifferentialValidator that handle the differences between the ASTs.

The basic implementation (FamixModelComparator) does a strict comparison (no differences allowed), but it has methods for accepting some differences:

#ast: node1 acceptableDifferenceTo: node2: If for some reason the difference in the nodes is acceptable, this method must return true and the comparison will restart from the parent of the two nodes as if they were the same.
#ast: node1 acceptableDifferenceTo: node2 property: aSymbol. This is for property comparison (eg. the name of an entity), it should return nil if the difference in value is not acceptable and a recovery block if it is acceptable. Instead of resuming from the parent of the nodes, the comparison will resume from an ancestor for which the recovery block evaluates to true.

Generating a visitor infrastructure for a given meta-model

Feb 26, 2025

Nicolas Anquetil

Moose expert

This post is part of a serie dedicated to Famix Tools

Once we have a model of a program in Famix, we often find ourselves wanting to ¨ go through it” to apply some systematic analysis or modification. For example one could want to export the model as source-code https://github.com/moosetechnology/FAMIX2Java.

The Visitor design pattern is well adapted for these tasks. Note that the double-dispatch mechanism, which is an integral part of the visitor pattern, may also be useful to have entity specific actions even if one does not want to visit the entire model.

The Visitor pattern requires:

an accept: aVisitor method in every entity of the meta-model (eg.: in FamixJavaClass, FamixJavaMethod, FamixJavaAttribute,…)
visitFamixXYZ: aFamixXYZ for all entites to be visited, in the visitor class
the accept: methods invoke the correct visitFamixXYZ: method of the visitor depending on the class it is implemented in
the visitFamixXYZ: by default recursively visits all the “children” of the entity being visited (eg.: visitFamixJavaClass: should trigger the visit of the attributes and methods of the class)

For large meta-models, this can be cumbersome to implement as there may be many kinds of entities (43 for FamixJava) and the work is very repetitive. The tool FamixVisitorCodeGenerator can do all the work for you, automatically.

FamixVisitorCodeGenerator

Taking advantage of the meta-description of the Famix entities and the reflective nature of Pharo, it generates the accept: and visitFamixXYZ: for all entities of a meta-model.

Usage exmaple:

FamixVisitorCodeGenerator new
  package: 'Famix-Java-Entities' visitorClass: FamixJavaVisitor .

or for a FAST meta-model:

FamixVisitorCodeGenerator new
  package: 'FAST-Java-Entities' visitorClass: FASTJavaVisitor.

The tool needs an empty visitor class (or trait) created by the user (FamixJavaVisitor in the example above), and a Pharo package containing the Famix classes of a meta-model (Famix-Java-Entities in the example above). From this it will:

create an accept: method in all the classes in the given Famix package;
the accept: methods are created as extensions made by the package of the visitor;
the accept: methods invoke the correct visitFamixXYZ: depending on the class they are implemented in.
a setter method allows to skip this part: generateAccepts: false
the visitFamixXYZ: methods are created in the visitor class (or trait) for a “maximal visit” (see below).

Visiting methods

For a friendlier visitor, it is convenient that the visitor methods reproduce the inheritance hierarchy of the entities. For example, if FamixJavaAttribute and FamixJavaParameter both inherit from FamixJavaVariable entity, it is convenient that visitFamixJavaAttribute: and visitFamixJavaParameter: both call visitFamixJavaVariable: so that a common behaviour for the visit can be implemented only once.

Since Famix heavily relies on traits to compose meta-models, the same idea applies to used traits. For example many entities implements FamixTSourceEntity to get a sourceAnchor. For all these entities, and if we need a generic behavior based on the source anchor, it is convenient that all the visitXYZ: methods call visitTSourceEntity: where the common behavior can be implemented.

Finally it might be convenient that the visitXYZ: methods, visit all the entites related to the one we are visiting. For example when visiting a FamixJavaVariable, it might be interesting to recursively visit its declaredType.

All these conditions defines what we call a “maxium” visit of the meta-model.

”Maximum” Visit

As described above, the maximum visitXYZ: methods will trigger a resursive visit of many things (super-class visit method, used traits visit method, all entities related to the one being visited).

One down side of this is that a maximum visit is not a viable one in Famix because all relationships are bi-directionnal, causing infinite loops in the visit. For example, if in a FamixJavaVariable we recursively visit its declaredType, then in the FamixJavaTType we will recursively visit the typedEntities including the one we just came from.

There could be various solution to this problem:

implement a memory mechanism in the visitor to remember what entities were already visited and do not go back to them.
do not visit all relationships of an entity, but only its “children” relationship
let the user handle the problem

For now the tool rely on the third solution. If an actual visitor inherits (or use) the generated visitor, it must take care or redefining the visit methods so that it does not enter in an infinite loop (implementing any of the two other solutions)

In this sense, the maximum visit methods can be viewed as cheat sheets, showing all the things that one could do when visiting an entity.

In the future we might implement the second solution (visit only children) as an option of the visitor. For now it is important to remember that the generated visitor cannot be used as is. Methods must be redefined to get a viable visitor.

class vs. trait visitor

The natural action would be to define the visitor as aclass from which all required actual visitors will inherit. However, because visitors are so handy to go through the entire model, we discovered that we needed a lot of them (eg. visitors to create a copy of a model, to re-export a model) and they sometimes need to inherit from some other class.

As such, we rather recommend to create the maximal visitor as a trait that can be used by all actual visitors. This make no difference for the FamixVisitorCodeGenerator and it might prove very useful.

Generate a class diagram visualization for a meta-model

Sep 26, 2023

Nicolas Anquetil

Moose expert

This post is the first in a serie dedicated to Famix Tools

When creating or studying a meta-model, it is often convenient to be able to “see” it as a whole.

UML looks like a natural solution for this.

So in the past we had a tool to create UML diagrams of the meta-models through PlantUML (a small language and a tool to generate UML diagrams). The post Generate a plantUML visualization for a meta-model explained how to use this tool

But the tool had some limitations, one of which was that it was not easy to add a different backend than PlantUML.

Therefore, inspired by the previous tool, we redesigned a new one, FamixUMLDocumentor, with a simpler API and the possibility to add new backends.

Simple Use

We illustrate the use with the same Coaster example already used previously. You can also experiment with FDModel, a small meta-model used for testing.

You can create a PlantUML script for a UML class of your metamodel with:

FamixUMLDocumentor new
  model: CCModel ;
  generate ;
  exportWith: (FamixUMLPlantUMLBackend new).

The result will be a PlantUML script that you can paste into https://plantuml.org/ to get this UML class diagram:

Generated UML class of the Coaster meta-model {: .img-fluid}

FamixDocumentor API

The API for the documenter is as follow:

model: — adds a meta-model to export. Several meta-models can be exported jointly by adding them one after the other. By default each meta-model is automatically assigned a color in which its entities will be drawn.
model:color: — same as previous but manually assign a Color to the meta-model.
onlyClasses: — specifies a list of classes to export. It can replace the use of model:.
excludeClasses: — specifies a list of classes to exclude from the export. Typically used with model: to remove from the UML some of the meta-model’s classes. Can also be used to exlude “stub” classes (see beWithStubs).
beWithStubs — Indicates to also export the super-classes and used traits of exported classes, even if these super-classes/traits or not part of the meta-models. These stubs have an automatically selected color different from the meta-models displayed.
beWithoutStubs — opposite of the preceding. This is the default option.
generate — creates an internal representation of a UML class diagram according to the configuration created with the preceding messages.
exportWith: — exports the internal representation with the “backend” given (for example: FamixUMLPlantUMLBackend in the example above)

FamixUML Backends

The backend is normally called by the FamixUMLDocumentor but can be called manually. For example, the image above can be exported in a PlantUML script with:

documentor := FamixUMLDocumentor new.
documentor
    model: CCModel ;
    generate.
FamixUMLPlantUMLBackend new export: documentor umlEntities.

(Compare with the example given above)

Backends have only one mandatory method:

export: — Exports the collection of umlEntities (internal representation) in the format specific to the backend.

New backends can be created by subclassing FamixUMLAbstractBackend.

There is a FamixUMLRoassalBackend to export the UML diagram in Roassal (visible inside Pharo itself), and a FamixUMLMermaidBackend to export in Mermaid format (similar to PlantUML).

There is a FamixUMLTextBackend that outputs the UML class diagram in a textual form. By default it returns a string but this can be changed:

toFile: — Instead of putting the result in a string, will write it to the file whose name is given in argument.
outputStream: — specifies a stream on which to write the result of the backend.

FamixUMLPlantUMLBackend and FamixUMLMermaidBackend are subclasses of this FamixUMLTextBackend (therefore they can also export to a file).

Automatic meta-model documentation generation

Jul 19, 2021

Benoit Verhaeghe

Moose expert

When you are developing with Moose everyday, you know how to create an excellent visualization of your meta-model. But, we have to open a Pharo image, and it is hard to share it during a presentation. Often, we made one UML of the meta-model, and then… we forget to update it. Then, when sharing with others, you have to say that the UML is not correct but it is ok???.

In my opinion, this is super bad. Thus, I decided to have a look at GitHub Actions to update my UML automatically.

In the following, I present how to update GitHub Actions to add UML auto-generation. I use the Coaster project as an example. Please consider reading the blog post about using GitHub action with Pharo.

Configure SmalltalkCI

The first step is to configure SmalltalkCI for your project. To use it, we need to create two files: .smalltalk.ston, and the GitHub actions: .github/workflows/ci.yml.

The .smalltalk.ston file is used to configure the CI. It is written in the Ston file format and configures how to load the Pharo project and how to test it. In our case, the Coaster project does not have tests 😱, so we set that the CI does not fail even if no tests are ran.

The final file can be found in the Coaster project.

SmalltalkCISpec {
  #loading : [
    SCIMetacelloLoadSpec {
      #baseline : 'Coaster',
      #directory : 'src',
      #load : [ 'default' ],
      #platforms : [ #pharo ],
      #onConflict : #useIncoming,
      #onUpgrade : #useIncoming
    }
  ],
  #testing : {
    #failOnZeroTests : false
  }
}

The second file, .github/workflows/ci.yml, is used by GitHub when running the CI. We describe in comments the main steps:

# Name of the project in the GitHub action panel
name: CI

# Execute the CI on push on the master branch
on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        # Use Moose 9 that includes our visualization tool
        smalltalk: [Moose64-9.0]
    name: ${{ matrix.smalltalk }}
    steps:
        # checkout the project
      - uses: actions/checkout@v2

        # Prepare the CI - download the correct VM :-)
      - uses: hpi-swa/setup-smalltalkCI@v1
        with:
          smalltalk-image: ${{ matrix.smalltalk }}

        # Use the CI - always better to run test
      - run: smalltalkci -s ${{ matrix.smalltalk }}
        shell: bash
        timeout-minutes: 15

Create the UML file

Once the main files are created, we can configure the CI also to create the UML file. To do so, we will use the plantUML visualization tool.

We add a new step in the .github/workflows/ci.yml file as a first step. It consists of executing the FamixMMUMLDocumentor on the meta-model we want to document.

- name: Build meta-model plantuml image
  run: |
    $SMALLTALK_CI_VM  $SMALLTALK_CI_IMAGE eval "'coaster.puml' asFileReference writeStreamDo: [ :stream | stream nextPutAll:  (FamixMMUMLDocumentor new model: CCModel; beWithStub; generatePlantUMLModel) ]."

This new step creates the coaster.puml file in the $HOME folder of the GitHub action. Then, we use a new action that creates the coaster.png file.

- name: Generate Coaster PNG Diagrams
  uses: cloudbees/plantuml-github-action@master
  with:
    args: -v -tpng coaster.puml

Nice 😄, we have the png file generated by the GitHub action.

Upload the UML

Finally, you can upload the UML png as an artifact of the Github action or upload it somewhere else. Here, I present how to publish it to a new branch of your repository. Then, we will see how to show it in the Readme of the main branch.

The goal of this step is to automatically update the documentation for end-users.

First, we create a new directory where we put the UML png file.

- name: Move artifact
  run: |
    mkdir doc-uml
    mv *.png doc-uml

Then, we configure this directory as a new git repository.

- name: Init new repo in doc-uml folder and commit generated files
  run: |
    cd doc-uml/
    git init
    git add -A
    git config --local user.email "action@github.com"
    git config --local user.name "GitHub Action"
    git commit -m 'update doc'

This new repository includes only the documentation we generated. The final step is to push this into a new branch of our project.

Because we do not care about the history of our meta-model UML files here, we will force push the repository. But creating more intelligent scripts is possible.

To do so, we use the ad-m/github-push-action GitHub action.

# Careful, this can kill your project
- name: Force push to destination branch
  uses: ad-m/github-push-action@v0.5.0
  with:
    # Token for the repo. Can be passed in using $\{{ secrets.GITHUB_TOKEN }}
    github_token: ${{ secrets.GITHUB_TOKEN }}
    force: true
    # Destination branch to push changes
    branch: v1/doc
    # We need to push from the folder where files were generated.
    # Same as where the new repo was initialized in the previous step
    directory: ./doc-uml

BE CAREFUL; if you incorrectly set the branch argument, you might delete your project.

When used, this action pushes the UML files in the v1/doc branch. The v1/doc of the Coaster project is created here.

Finally, we add the image of the UML files in the Readme of the main project. For the Coaster project, we modified the Readme and added:

![Coaster meta-model png](https://raw.githubusercontent.com/badetitou/CoastersCollector/v1/doc/coaster.png)

The URL follows the following pattern: https://raw.githubusercontent.com/:owner:/:repo:/:branch:/:file:. The final .github/workflows/ci.yml file is here.

Conclusion

That’s it 😄 Now, at every commit, the CI will update the png files used in the Readme of the project, and thus, the documentation is always up-to-date.

Generate a plantUML visualization for a meta-model

Jun 4, 2021

Théo Lanord

Trainee / Apprentice engineer

Note

This post describes a tool that has been replaced by a new FamixUMLDocumentor. The new tool is described in another post.

Old post content

When you are interested in a meta-model of which you are not the creator, it is sometimes difficult to understand it only using the declarations in the code. It would be best if you could actually visualize it in a different way. What better way to go back to a very efficient meta-model visualization tool: UML.

In this blog, I will show you how to generate plantUML code from a generated meta-model. For that, I will take the example of the evolution of the meta-model on coasters:

There is no need to do these posts to understand this one. I would even say that this is precisely the subject: to study an unknown meta-model.

Prerequisite and details

First of all, and if it has not already been done, do not forget to download and generate the meta-models using its generator. For example, for the basic Coasters collection, the code is available on Coaster GitHub repository and it can be generate with:

CoasterCollectorMetamodelGenerator generate

FamixMMUMLDocumentor, the tool I am going to present to you, is based on the generated meta-model. Therefore, it is particularly suitable for models with subMetamodels (Cf. beWithStub option). It is important to note that another tool, based on the meta-model builder, exists : FmxMBPlantTextVisitor. It can be interesting if you need to display the compositions.

I would also like to make one last remark, most of the information given in this post can be found in the comment of the FamixMMUMLDocumentor class. Finally, there is the plantUML server to run your plantUML code directly on the web. So let’s continue and generate our visualizations! 😄

Global approach

Generation on the whole meta-model

Let’s say we know that there is a meta-model on coasters whose builder is CoasterCollectorMetamodelGenerator. Since we need the generated model and not the builder, we will look at the prefix defined in CoasterCollectorMetamodelGenerator class >> #prefix and deduce the model name, which consists of the model prefix followed by the word Model.

In this case, for CoasterCollectorMetamodelGenerator, the model is called CCModel. From here, we have all the elements to generate the plantUML code associated with the model via the following code:

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLModel.

The generation is done by instantiating a FamixMMUMLDocumentor for which we provide the model (model:) and ask for the complete generation for this last one (generatePlantUMLModel).

UML representation of Coaster meta-model

We can now compare the generated UML representation to the basic one that helped create the generator or that has been used to generate the generator 😄 (Cf. Model your Fame/Famix meta-model using Graphical Editors).

"coasters UML"

We can observe a UML diagram that is almost identical. Only CCModel is additional. However, generation options allow solving this problem (and many others).

Generation options

generatePlantUMLModelWithout

Indeed, it is possible to ask to generate the plantUML code without a defined collection of entities. For example, if you do not want the CCModel to appear.

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLModelWithout: { CCModel }.

UML representation (option Without) of Coaster meta-model

It is important to note that it is necessary to give the entities themselves and not their names. That is to say that it is necessary to add their prefix. For example, the entity associated with the name Coaster is CCCoaster.

generatePlantUMLWith

It is also possible to do the opposite. That is to say to select only the entities to generate.

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLWith: { CCCoaster . CCCreator . CCBrewery }.

UML representation (option With) of Coaster meta-model

This can be useful if you are interested in certain entities.

beWithStub

Finally, there is one last exciting possibility. If we take the case of the evolution of the coasters meta-model extended in terms of creators Connecting/Extending meta-models.

Extended Coaster meta-model

Let’s generate the plantUML code on the meta-model and observe.

FamixMMUMLDocumentor new
 model: CCEModel ;
 generatePlantUMLModelWithout: { CCEModel }.

UML representation of Extended Coaster meta-model

We can say that the representation is deceiving, but it is only a representation of what is declared in the meta-model. However, the meta-model has a subMetamodel, so we have to look for these dependencies in it. For this, there is the beWithStub option.

FamixMMUMLDocumentor new
 beWithStub;
 model: CCEModel ;
 generatePlantUMLModelWithout: { CCEModel . MooseModel }.

UML representation with stub of Extended Coaster meta-model

We can see that Event inherits from an external class Creator, coming from the subMetamodel CoasterCollectorMetamodelGenerator.

It would indeed be interesting to generate the subMetamodel view as well in order to have a better overall view, maybe an improvement track?

Output to a file

Each option is available in text or file output via the following methods:

generatePlantUMLModel / generatePlantUMLModelFile:
generatePlantUMLModelWithout: / generatePlantUMLModelFile:without:
generatePlantUMLWith: / generatePlantUMLFile:with:

Application to a larger model

To finalize this post, we will generate a larger meta-model that aggregates all the notations available in the tool. To do this, I chose FASTModel, a method syntax analysis meta-model, available with this moosetechnology GitHub repository.

FamixMMUMLDocumentor new
 beWithStub;
 model: FASTModel;
 generatePlantUMLModelWithout: { FASTModel . MooseModel }.

UML representation with stub of FASTCore

Notations

In summary, we have 5 specific notations:

Internal entity notations:
- Class: Black C on white background
- Trait: Black T on grey background
External entity notations:
- Class: Black C on yellow background with External label
- Trait: Black T on yellow background with External label
Use of traits: Dashed arrow

The rest of the notations follows the UML standard.

Conclusion

In this post, we have seen how to visualize a meta-model using FamixMMUMLDocumentor. This feature is handy for understanding complex meta-models and allows (almost) automatic documentation.