Blog

Towards analyzing TypeScript with Moose

Jun 23, 2021

Christopher Fuhrman

Moose herder

TypeScript is a more and more popular programming language, and so it would be great if we could analyze TypeScript projects using Moose. At the time of writing, no meta-model (or importer) exists for the TypeScript language in Moose. First, what are the pieces of the puzzle needed to analyze TypeScript with Moose? Before we consider TypeScript, let’s look at how things work with Java:

VerveineJ is the importer that can generate models of Java files, allowing us to do analyses in Pharo/Moose.

What’s needed to analyze TypeScript?

If we want to do the same thing for TypeScript, we would need:

• an equivalent of VerveineJ (importer) for TypeScript files,
• a Famix model of TypeScript.

Creating a parser and importer for TypeScript is no small task, but TypeScript is a popular environment and we can use ts-morph to facilitate the navigation of the TypeScript AST. There’s also a very cool visualization of TypeScript ASTs, which will be useful for understanding and debugging.

Designing a new meta-model for TypeScript is definitely not trivial, because it requires a deep understanding of the language. On the other hand, once a meta-model exists, it’s easy to generate using FamixNG domain-specific language.

Pragmatically speaking, do we need a perfect model of TypeScript to analyze it?

abap2famix

“All models are wrong, but some are useful.” —maybe not George Box

By searching the web for TypeScript and Moose, I discovered a GitHub project called pascalerni/abap2famix. It is an ABAP importer (written in TypeScript) that models ABAP projects using FAMIX 3.0 (compatibility meta-model for Java). Java and ABAP are indeed different languages, but perhaps the differences are not so important if we want to do some static analysis? Seems like a pragmatic approach!

Looking at the node packages used by abap2famix I discovered famix, a TypeScript implementation of Famix, which facilitates creating FAMIX 3.0 entities from TypeScript. Its source is at pascalerni/famix, and I could see that much of it was generated, e.g., in class.ts there’s proof it was not written by hand:

// automatically generated code, please do not change

import {FamixMseExporter} from "../../famix_mse_exporter";
import {Type} from "./../famix/type";

export class Class extends Type {
  ...
}

How was this code generated? The answer is the fork of FameJava at pascalerni/FameJava, namely the Famix30Codegen.java file. The original FameJava was used to generate the Java API for use with FAMIX 3.0 metamodel. This fork generates (via Java) a TypeScript API. Clever and useful!

Drivers, start your engines!

So, what if we try to create an importer using ts-morph and the famix packages that will model TypeScript programs in a Java metamodel? As a first try, model only the object-oriented elements of TypeScript, such as classes, methods, attributes, etc.

This is actually the project I proposed to students in an advanced-topics software design course at my university during the winter of 2021.

Results

Several teams set out to achieve this goal, although none of the students had ever done parsing or Pharo before. Many were familiar with node and TypeScript.

I’m happy to say that they all were successful (in varying degrees) in writing an importer for TypeScript that allowed analyses to be done in Moose and Pharo, and their results are all on GitHub:

Team 1 | Team 2 | Team 3 | Team 4

Here are some visualizations produced by Team 4 using Roassal on models loaded into Moose.

The first shows the Weighted Method Count (sum) for classes in several TypeScript projects. The cyclomatic complexity values were calculated using another npm package (ts-complex) in the TypeScript importer:

The following chart shows distributions of Cyclomatic Complexities of methods for various classes in the prisma project:

Limits of modeling TypeScript in a Java metamodel

Here are some of the obvious things in TypeScript (Javascript) that don’t quite fit into a Java model:

• Functions can exist in the global namespace. A workaround proposed by one team was to create a “Global” class in the Java model, and just put functions there as static methods.
• Functions can exist in methods, but maybe this is possible to model in a newer meta-model for Java that supports lambdas. The API from pascalerni/famix supports an older meta-model for Java.
• string, number, any are types in TypeScript, but they do not really map to primitive types or classes in Java.
• TypeScript doesn’t have packages like Java, although it does have ways to specify namespaces and avoid naming conflicts.

Conclusion

Even though a formal model in TypeScript doesn’t (yet) exist in Famix, it’s possible to perform useful analyses of TypeScript using the FAMIX 3.0 (Java) metamodel, thanks to packages, tools and APIs developed and reused in the npm and Moose communities.

Photo credit: “patchwork beads” (CC BY-SA 2.0) by various brennemans

Tags:

• story

Generate a plantUML visualization for a meta-model

Jun 4, 2021

Théo Lanord

Trainee / Apprentice engineer

Note

This post describes a tool that has been replaced by a new FamixUMLDocumentor. The new tool is described in another post.

Old post content

When you are interested in a meta-model of which you are not the creator, it is sometimes difficult to understand it only using the declarations in the code. It would be best if you could actually visualize it in a different way. What better way to go back to a very efficient meta-model visualization tool: UML.

In this blog, I will show you how to generate plantUML code from a generated meta-model. For that, I will take the example of the evolution of the meta-model on coasters:

• Coasters collection
• Connecting/Extending meta-models

There is no need to do these posts to understand this one. I would even say that this is precisely the subject: to study an unknown meta-model.

Prerequisite and details

First of all, and if it has not already been done, do not forget to download and generate the meta-models using its generator. For example, for the basic Coasters collection, the code is available on Coaster GitHub repository and it can be generate with:

CoasterCollectorMetamodelGenerator generate

FamixMMUMLDocumentor, the tool I am going to present to you, is based on the generated meta-model. Therefore, it is particularly suitable for models with subMetamodels (Cf. beWithStub option). It is important to note that another tool, based on the meta-model builder, exists : FmxMBPlantTextVisitor. It can be interesting if you need to display the compositions.

I would also like to make one last remark, most of the information given in this post can be found in the comment of the FamixMMUMLDocumentor class. Finally, there is the plantUML server to run your plantUML code directly on the web. So let’s continue and generate our visualizations! 😄

Global approach

Generation on the whole meta-model

Let’s say we know that there is a meta-model on coasters whose builder is CoasterCollectorMetamodelGenerator. Since we need the generated model and not the builder, we will look at the prefix defined in CoasterCollectorMetamodelGenerator class >> #prefix and deduce the model name, which consists of the model prefix followed by the word Model.

In this case, for CoasterCollectorMetamodelGenerator, the model is called CCModel. From here, we have all the elements to generate the plantUML code associated with the model via the following code:

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLModel.

The generation is done by instantiating a FamixMMUMLDocumentor for which we provide the model (model:) and ask for the complete generation for this last one (generatePlantUMLModel).

We can now compare the generated UML representation to the basic one that helped create the generator or that has been used to generate the generator 😄 (Cf. Model your Fame/Famix meta-model using Graphical Editors).

We can observe a UML diagram that is almost identical. Only CCModel is additional. However, generation options allow solving this problem (and many others).

Generation options

generatePlantUMLModelWithout

Indeed, it is possible to ask to generate the plantUML code without a defined collection of entities. For example, if you do not want the CCModel to appear.

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLModelWithout: { CCModel }.

It is important to note that it is necessary to give the entities themselves and not their names. That is to say that it is necessary to add their prefix. For example, the entity associated with the name Coaster is CCCoaster.

generatePlantUMLWith

It is also possible to do the opposite. That is to say to select only the entities to generate.

FamixMMUMLDocumentor new
 model: CCModel ;
 generatePlantUMLWith: { CCCoaster . CCCreator . CCBrewery }.

This can be useful if you are interested in certain entities.

beWithStub

Finally, there is one last exciting possibility. If we take the case of the evolution of the coasters meta-model extended in terms of creators Connecting/Extending meta-models.

Let’s generate the plantUML code on the meta-model and observe.

FamixMMUMLDocumentor new
 model: CCEModel ;
 generatePlantUMLModelWithout: { CCEModel }.

We can say that the representation is deceiving, but it is only a representation of what is declared in the meta-model. However, the meta-model has a subMetamodel, so we have to look for these dependencies in it. For this, there is the beWithStub option.

FamixMMUMLDocumentor new
 beWithStub;
 model: CCEModel ;
 generatePlantUMLModelWithout: { CCEModel . MooseModel }.

We can see that Event inherits from an external class Creator, coming from the subMetamodel CoasterCollectorMetamodelGenerator.

It would indeed be interesting to generate the subMetamodel view as well in order to have a better overall view, maybe an improvement track?

Output to a file

Each option is available in text or file output via the following methods:

• generatePlantUMLModel / generatePlantUMLModelFile:
• generatePlantUMLModelWithout: / generatePlantUMLModelFile:without:
• generatePlantUMLWith: / generatePlantUMLFile:with:

Application to a larger model

To finalize this post, we will generate a larger meta-model that aggregates all the notations available in the tool. To do this, I chose FASTModel, a method syntax analysis meta-model, available with this moosetechnology GitHub repository.

FamixMMUMLDocumentor new
 beWithStub;
 model: FASTModel;
 generatePlantUMLModelWithout: { FASTModel . MooseModel }.

Notations

In summary, we have 5 specific notations:

• Internal entity notations:
  • Class: Black C on white background
  • Trait: Black T on grey background
• External entity notations:
  • Class: Black C on yellow background with External label
  • Trait: Black T on yellow background with External label
• Use of traits: Dashed arrow

The rest of the notations follows the UML standard.

Conclusion

In this post, we have seen how to visualize a meta-model using FamixMMUMLDocumentor. This feature is handy for understanding complex meta-models and allows (almost) automatic documentation.

Tags:

• Famix-tools

Connecting/Extending meta-models

May 15, 2021

Benoit Verhaeghe

Moose expert

Sometimes, a meta-model does not have all the information you want. Or, you want to connect it with another one. A classic example is linking a meta-model at an abstract level to a more concrete meta-model.

In this blog post, I will show you how to extend and connect a meta-model with another (or reuse a pre-existing meta-model into your own meta-model). We will use the Coaster example.

Extending the Coaster meta-model

The Coaster meta-model is super great (I know… it is mine 😄 ). Using it, one can manage its collection of coasters.

However, did you notice that there is only one kind of Creator possible: the brewery. It is not great because some coasters are not created by breweries but for events. My model is not able to represent this situation. So, there are two possibilities: I can fix my meta-model, or I can extend it with a new concept. Here, we will see how I can extend it.

As presented in the above figure, we add the Events concept as a kind of Creator.

Configuration

As a first step, we need the original Coaster meta-model generator loaded in our image. We can download it from my Coaster GitHub repository.

You should have a named CoasterCollectorMetamodelGenerator in your image. This is the generator of the original meta-model. We will now create another generator reusing the original one.

First, we create a new generator for our extended meta-model.

FamixMetamodelGenerator subclass: #CoasterExtendedMetamodelGenerator
    instanceVariableNames: ''
    classVariableNames: ''
    package: 'CoasterCollector-ExtentedModel-Generator'

Then, we link our generator with the original one. To do so, we will use the submetamodels feature of the generator. We only have to implement the #submetamodels method in the class side of our new generator. This method should return an array including the generators of the submetamodels that we want to reuse.

CoasterExtendedMetamodelGenerator class >> #submetamodels
    ^ { CoasterCollectorMetamodelGenerator }

Finally, as for a classic meta-model generator, we define a package name and a prefix.

CoasterExtendedMetamodelGenerator class >> #packageName
    ^ #'CoasterExtended-Model'

CoasterExtendedMetamodelGenerator class >> #prefix
    ^ #'CCE'

Define entities

Creating new concepts in the new meta-model is done following the same approach as for classic meta-model generator. In our example, we add the Event class. Thus, we create the method #defineClasses with the new entity.

CoasterExtendedMetamodelGenerator >> #defineClasses
    super defineClasses.
    event := builder newClassNamed: #Event.

Declare entity of submetamodels

To extend the original meta-model, we first need to identify the entities of the original meta-model we will extend. In our case, we only extend the Creator entity. Thus, we declare it in the #defineClasses method. To do so, we use the method #remoteEntity:withPrefix:. The prefix is used to allow multiple entities coming from different submetamodels but with the same name.

CoasterExtendedMetamodelGenerator >> #defineClasses
    super defineClasses.
    event := builder newClassNamed: #Event.

    "Remote entities"
    creator := self remoteEntity: #Creator withPrefix: #CC

We refer to a remote entity by sending #remoteEntity:withPrefix: to self and not using the builder. Indeed, the entity is already created.

Using remote entity

Once the declaration done, one can use the remote entities as classic entities in the new generator. In our example, we will create the hierarchy between Creator and Event.

CoasterExtendedMetamodelGenerator >> #defineHierarchy
    super defineHierarchy.
    event --|> creator

Generate

Once everything is defined, as for classic generator, we generate the meta-model. To do so, execute in a playground:

CoasterExtendedMetamodelGenerator generate

The generation creates a new package with the Event entity. It also generates a class named CCEModel used to create an instance of our extended meta-model.

Use the new model

It is now possible to use the new meta-model with the Event concept. For instance, one can perform the following script in a playground to create a little model.

myExtendedModel := CCEModel new.
myExtendedModel add: (CCBrewery new name: 'Badetitou'; yourself).
myExtendedModel add: (CCEEvent new name: 'Beer party'; yourself)

Connecting meta-models

We saw that one can extend a meta-model by creating a new one based on the pre-existing entities. It is also possible to connect two existing meta-models together.

To do so, let’s assume we have two existing meta-models to be connected. As an example, we will connect our coaster meta-model, with the world meta-model. The world meta-model aims to represent the world, with its continent, countries, regions and cities.

The world meta-model

We will not detail how to implement the world meta-model. But the generator is available in my GitHub repository. The figure below illustrates the meta-model.

Connecting world meta-model with Coaster meta-model

Our goal is to connect the coaster meta-model with the world meta-model. To do so, we will connect the country concepts of each meta-model.

As a first step, you should install both the coaster meta-model and the world meta-model. Again, both are available in my GitHub repository.

Then, we create a new meta-model generator that will perform the connection.

FamixMetamodelGenerator subclass: #ConnectMetamodelGenerator
    instanceVariableNames: ''
    classVariableNames: ''
    package: 'Connect-Model-Generator'

Declare submetamodels

To connect together the two meta-models, we must first declare them in our connector meta-model. To do so, we define the #submetamodels method.

ConnectMetamodelGenerator class >> #submetamodels

    ^ { WorldMetamodelGenerator . CoasterCollectorMetamodelGenerator }

And, as for every meta-model generator, we define a prefix and a package name.

ConnectMetamodelGenerator class >> #packageName

    ^ #'Connect-Model'

ConnectMetamodelGenerator class >> #submetamodels

    ^ #'CM'

Connect remote entities

Before creating the connection, we must declare, in the new meta-model, the entities that will be contected. To do so, we declare them as remoteEntity.

ConnectMetamodelGenerator >> #defineClasses
    super defineClasses.
    coasterCountry := self remoteEntity: #Country withPrefix: #CC.
    worldCountry := self remoteEntity: #Country withPrefix: #W

Then, it is possible to connect the two entities as classic one.

ConnectMetamodelGenerator >> #defineRelations
    super defineRelations.
    coasterCountry - worldCountry

Build a model with two connected submetamodels

Once the generator is created, we can generate the connection by generating the new meta-model. To do so, execute in a playground:

ConnectMetamodelGenerator generate

Then, it is possible to create a model with all the entities and to link the two meta-models. In the following, we present a script that create a model.

"create the entities"
coaster1 := CCCoaster new.
coaster2 := CCCoaster new.
coaster3 := CCCoaster new.

coasterFranceCountry := CCCountry new name: #'France'; yourself.
coasterFranceCountry addCoaster: coaster1.
coasterFranceCountry addCoaster: coaster2.

coasterGermanyCountry := CCCountry new name: #'Germany'; yourself.
coasterGermanyCountry addCoaster: coaster3.

wFranceCountry := WCountry new name: #'France'; yourself.
wGermanyCountry := WCountry new name: #'Germany'; yourself.

continent := WContinent new name: #Europe; yourself.
continent addCountry: wFranceCountry.
continent addCountry: wGermanyCountry.

"connect CCountries to WCountries"
coasterFranceCountry country: wFranceCountry.
coasterGermanyCountry country: wGermanyCountry.

"put all entities into the same model"
connectedModel := CMModel new.
connectedModel addAll:
    { coaster1. coaster2 . coaster3 .
    coasterFranceCountry . coasterGermanyCountry .
    wFranceCountry . wGermanyCountry . continent }.

Based on the preceding model, it is possible to create query that will request the coaster and the world meta-model. For instance, the following snippet count the number of coasters by country in the Europe continent:

europe := (connectedModel allWithType: WContinent)
    detect: [ :continent | continent name = #Europe ].
(europe countries collect: [ :eCountry |
    eCountry name -> eCountry country coasters size ]) asDictionary

The coutry: and coutry methods are accessors that allow to set and recover one CCCountry (resp. WCountry) into a WCountry (resp. CCCountry). The accessors names are the same in both classes and were generated automatically from the declaration of the relationship in defineRelations (this is normal behaviour of the generator, not specific to using sub-models.

Conclusion

In this post, we saw how one can extend and connect meta-models using Famix Generator. This feature is very helpfull when you need to improve a meta-model without modifying it directly. If you need more control on the generated entities (e.g., name of the relations, etc.), please have a look at the create meta-model wiki page.

Tags:

• meta-model

How to build a new Moose tool: The MooseInspector

May 4, 2021

Sebastian Jordan

Moose developer

How to build a new Moose tool: The MooseInspector

To create a new Moose Tool, you must create a child class of MiAbstractBrowser. This abstract class contains the basic infrastructure to all Moose browsers. It provides a toolbar with: buttons to inspect and propagate the current selection; Radio buttons to choose a reception mode; and a help button that shows the class comment for each browser.

Also, it provides the logic to connect the browser to the Moose bus.

So, let us get started. We will create a “Moose Inspector”. It would be like the Pharo’s inspector but as a Moose browser. Firstly, we create the subclass as following:

MiAbstractBrowser subclass: #MiInspectorBrowser
    instanceVariableNames: 'stInspector'
    classVariableNames: ''
    package: 'Moose-Core-Inspector'

As one can see, it has one instance variable which will hold an instance of Pharo’s inspector: StInspector.

Now, we must implement some basic methods. First let us implement initializePresenters method:

initializePresenters

    super initializePresenters.
    stInspector := self instantiate: StInspector.
    stInspector model: self model

We instantiate stInspector variable an instance of Pharo’s inspector. Then we set the inspector model to be the same as the browser model.

Now we are going to implement canReceiveEntity: method. This method returns a Boolean which tells us if the entities received on the bus are usable in this browser. As we are building an inspector all entities can be accepted. So, we are going to return true always.

canReceiveEntity: anEntity

    ^ true

Then, we must implement followEntity: method. This method is called when new entities are received from the bus. In this case, we only need update the inspector model with the new entity. This method has the responsibility of defining the behaviour of the browser when new entities arrives from the bus. This is part of the bus mechanism of MiAbstractBrowser. This method is called if canReceiveEntity: anEntity returns true.

followEntity: anEntity

    self model: anEntity.
    stInspector model: self model

Next, the miSelectedItem method tells the bus what to propagate (when the user hits the “Propagate” button). In this case we want to propagate the object selected in the last inspector page.

miSelectedItem

    | lastInspectorPage |
    lastInspectorPage := stInspector millerList pages last.
    ^ lastInspectorPage model inspectedObject

Now we have all the logic and we can define the layout of this new browser. Now in Spec, the framework used to buld GUi in Pharo, we can implement dynamic layouts. So, we create a initializeLayout method. In that method, we take the layout of the super class, which is the toolbar, and we will add the other presenter.

initializeLayout

  self layout: (SpBoxLayout newTopToBottom
    add: self class defaultSpec expand: false;
    add: stInspector;
    yourself)

And do not forget to call this at the end of initializePresenters.

initializePresenters

    super initializePresenters.
    stInspector := self instantiate: StInspector.
    stInspector model: self model.
    self initializeLayout

Finally, we can define which will be the default model on which the browser will open. This is in case the bus does not have any entities. We want the Moose models, so create on class side:

newModel

    ^ MooseModel root entities

Optionally, we can override class side methods title and windowSize. We are ready to go. All we must do now is to run MiInspectorBrowser open on a Playground. This will open our new browser.

How to add new tabs in the Moose Inspector Browser

The new browser is not effective yet. We want to add some custom tabs to inspect the entities in a more visual way. To do so we can add some custom inspector tabs. When it displays an object, the inspector looks for methods of that object that have the <inspectorPresentationOrder:title:> pragma. The method should return a spec presenter that will be included in the inspector’s tab.

We will migrate the old “Properties” tab that is found in MooseFinder. The code is in MooseObject>>#mooseFinderPropertiesIn: We only have to rewrite it using Spec framework and use the pragma <inspectorPresentationOrder:title:>. We will create a method in MooseObject called inspectorPropertiesIn.

inspectorPropertiesIn

    <inspectorPresentationOrder: 2 title: 'Properties'>
    ^ SpTablePresenter new
        items: self mooseInterestingEntity mooseDescription allPrimitiveProperties;
        sortingBlock: [ :x :y | x name < y name ];
        addColumn: (SpStringTableColumn title: 'Properties' evaluated: #name);
        addColumn: (SpStringTableColumn title: 'Value' evaluated: [ :each | (self mooseInterestingEntity mmGetProperty: each) asString ]);
        yourself

Because the method is defined in MooseObject, any subclass (MooseModel, MooseEntity, …) will have this tab when displayed in the inspector.

That it is! Now we run again: MiInspectorBrowser open and we will se that the new tab now appears.

Tags:

• infrastructure
• browser

Dependency Structure Matrix for a Java project using Moose

Apr 20, 2021

Christopher Fuhrman

Moose herder

Dependency Structure Matrix for a Java project using Moose

As an extension to Analyzing Java With Moose, in this post I will show how one can create a Design Structure Matrix (DSM) in Moose, in particular from a model of a Java project.

First, you need to generate and load an MSE file into Moose for a Java project. Refer to this post for those steps, which uses the Java code from from Head First Design Patterns.

Roassal (which is a visualization platform that’s part of Moose) has a visualization for DSM called RTDSM. It’s explained here, but with Pharo classes. How to use it with Moose on a Java model?

The key is in the dependency: block, which we define using a Moose Query with allClients. Open a Moose Playground and paste the following Pharo code:

| dsm classes |
dsm := RTDSM new.
classes := (MooseModel root first allModelClasses
  select: [ :c |
    c mooseName
      beginsWith:
        'headfirst::designpatterns::combining::decorator' ])
  reject: #isAnonymousClass.
"Avoid arbitrary ordering by sorting"
dsm objects: (classes asSortedCollection: [ :a :b | a name < b name]).
"Change the default label from asString which is very long"
dsm labelShapeX label text: #name.
dsm labelShapeY label text: #name.
"Moose Query equivalent to #dependentClasses for a Pharo class"
dsm dependency: #allClients.
dsm rotation: 270.
^dsm

This visualization may not be as powerful as IDEA’s DSM analysis or Lattix’s, but it’s open source and can be manipulated in Pharo.

In Moose Query, the opposite to allClients is allProviders.

Tags:

• browser
• story

Newer posts

Older posts