Blog

Manage rules using MooseCritics

Aug 8, 2022

Romain Degrave

Master degree intern in software analysis

Software projects often leave specific architectural or programming rules that are not checked by the off-the-shelf static analysis tools and linters.
But MooseCritics is now here to make such things easy!

Setting up for rule making

The first step to use this tool is of course to open its browser, findable in the Moose menu under the name Moose Critic Browser. As with every other tool of MooseIDE, we also need to propagate a model to give our tool entities to analyze. For this analysis, we will use a model of ArgoUML, an open-source Java project used in this wiki.

Rules and how to write them

Rules in MooseCritics are divided into two components: Context, and Condition.
A context is a collection of entities to specify the scope of our analysis. Using this, we are only executing our rules on the relevant entities for them.
Once we have a context, we add conditions to it, to verify the validity of every entity belonging to this context.

Let’s start building a few of those, to appreciate how easy and versatile this system can be!

Contexts

To begin, we will right-click on the root context, the root of our rules, doing nothing but passing the whole set of entities propagated into our browser. Then, clicking on “Add Context” will open a new window, in which we can write our first context.

As you can see, a context has three properties :

• Name: the name of our context
• Context Block: a code block, using as a parameter the collection given by the parent context, and that must return a collection of entities
• Summary: a quick explanation of the selection performed

In this case, the selection is very basic (keeping only the classes defined within our model), but any way of manipulating a collection (so long as it remains a collection) can be used to make a very specific choice of entities.
But for now, let’s keep things simple, and add a few more contexts to our root.

First, we select methods…

"Title:"
  'Methods'
"Context Block:"
  [ :collection | collection allMethods ]
"Summary:"
  'Every method in our model or called by a model entity.'

… and secondly attributes.

"Title:"
  'Attributes'
"Context Block:"
  [ :collection | collection allAttributes ]
"Summary:"
  'Every attributes in our model or accessed by a model entity.'

Once this is all done, we are met with this screen :

Conditions

Now that our contexts are set, we can write a few conditions for those.
To do so, right-clicking on our Model Classes context and choosing “Add condition” which will open a new interface to write our conditions.

The properties are almost identical to a context, but we now use a query to know whether or not an entity violates a rule.
This query will have as a parameter every entity of our context, one by one, and will add a violation to it if the query returns true.

Now, the most perceptive readers (all of my readers, no doubts 😄) will have noticed the two radio buttons; Pharo Code and Queries Browser.
We can indeed use a query built in the Queries Browser, and we will do so for the next one, to find God Classes.

This may not be an option for every kind of rule, especially the more complex ones, but conditions verifying several simple things can be easily designed, thanks to the Queries Browser.

Now that we saw all possibilities, time to write one more condition, this time for the methods :

"Title:"
  'Deprecated'
"Query Block:"
  [ :entity |
    entity annotationInstances notEmpty and: [
      entity annotationTypes
        anySatisfy: [ :a | a name = 'Deprecated' ] ] ]
"Summary:"
  'Deprecated methods, that should be removed or not used anymore.'

We are now all set, and all that remains to do is pressing the “Run” button in the bottom right corner, and look at the result of our analysis in the right pane, showing every violation found, on the format violatingEntity -> violatedCondition.

Now that we executed our rules, you can also have fun clicking on contexts and conditions to see that the left and right panels will change to match your selection, the left one showing the context, and the right one showing the violations of the selected condition, or the violations of every condition of the selected context.

Getting specific

We may also be a bit more specific, both on the condition side of things, but also when it comes to context.
For the conditions, our perceptive minds did not forget about the attributes, so we will write a condition for them too :

"Title:"
  'Directly accessed'
"Query Block:"
  [ :entity |
    entity accessors anySatisfy: [ :m |
      m isGetter not and: [ m isSetter not ] ] ]
"Summary:"
  'Every attribute accessed without the use of a getter or setter method.'

For a final rule, let’s work a bit more on our context. Let’s say we want to build a rule around getter methods, to verify that their cyclomatic complexity is equal to 1.
For that, we can start by making a new context, using the “Methods” context as its parent :

"Title:"
  'Getters'
"Context Block:"
  [ :collection |
    collection select: [ :m |
      (m name beginsWith: 'get') and: [ m isGetter ] ] ]
"Summary:"
  'Every getter method of our model, meaning :
    - Their name starts with 'get'
    - They have the property 'isGetter' set to true'

Once that sub-context has been created, we can give it a condition to verify ! Let’s do so right away, with our cyclomatic complexity example :

"Title:"
  'Cyclomatic Complexity > 1'
"Query Block:"
  [ :entity | entity cyclomaticComplexity > 1 ]
"Summary:"
  'A getter must have a cyclomatic complexity of one.'

We are now done with all of our rules. To get the result of our new conditions, you can use again the “Run” button, or, execute only the new ones by right-clicking on them and selecting the “Run condition” option.

Saving and loading the rules

Our work is now done, but we would like to be able to monitor the state of our project in the long run, and to simplify this, we can export and import sets of rules built with MooseCritics.

For that, by pressing the “Export rules” button, we can choose where we wish to save our rules. The loading works similarly and will restore the tree as it was when it was saved (if rules were already present, the imported rules are added after those).

Exporting violations

MooseCritics can also propagate those violations, in order to access them in the entities exporter, to be able to save those violations in a CSV file.
The exported selection will be the violations found in the right pane when using the propagate button.

Conclusion

MooseCritics enables us to verify the validity of our defined rules and puts us in the right direction to correct our mistakes by finding violations. Dividing our model into contexts allows us to make specific analyses while working on a large scale.
Even if most of the examples shown here are fairly simple, MooseCritics can represent complex structural rules using Famix properties and will surely make your life easier when it comes to software analysis using Moose. 😄

Tags:

• browser

Carrefour: The bridge between FAMIX and FAST

Jun 30, 2022

Ahmed Zaki BENNECER

Working on data flow analysis project for my internship

Introduction

To analyze software systems, the Famix meta-model provides enough abstraction to understand how models work.

However, when we are interested in details, the FAST (Famix AST) meta-model provides less abstraction and gives us more information about our model (for example expression statements, identifiers etc.).

In some situations, such as modernization/migration projects, we need the binding between the two meta models. And here Carrefour comes in!

Carrefour represents a two-way link between Famix and FAST, it allows one to navigate on the AST and at the same time return to the elements of FAMIX when needed.

Configuration

In this blog, we are going to use a simple snippet of code to simplify and grasp all necessary concepts we should know about FAST & Carrefour & Famix. Consider the MyClass class and the following methodAB method:

class MyClass {
  public int methodAB(int a, int b){
    if (a > b) {
      a = a + 2;
    } else {
      b = 1;
    }
    return b;
  }
}

Let’s prepare the ground for using Carrefour by generating the Famix model of the MyClass class using VerveineJ. Open a code editor and create a new MyClass.java file. Inside the Java file, we add the code above.

To generate the Famix Java model we use VerveineJ by running this command in the MyClass java file directory:

/path/to/VerveineJ/verveinej.sh -format json -o MyClass.json -anchor assoc -autocp ./ ./

PS: Note that Carrefour uses entities & associations as source anchor information, so make sure to add the option -anchor assoc.

Load Carrefour

In this section, we start from the MyClass Famix java model and we build the link between Famix and FAST using Carrefour. First, we need to install Carrefour, for example by running the following script on Moose Playground:

Metacello new
    githubUser: 'badetitou' project: 'Carrefour' commitish: 'v3' path: 'src';
    baseline: 'Carrefour';
    load

Then we import the MyClass model and pick the first class (which is the only class MyClass).

'/path/to/MyClass.json' asFileReference readStreamDo: [ :stream |
    model := FamixJavaModel new importFromJSONStream: stream
  ].
model rootFolder: '/path/to/MyClass/Directory/'.
method := model allModelClasses first.

Now, we call Carrefour to generate the AST (the figure below) and bind the newly created AST with Famix.

method generateFastAndBind

it’s recommended to use generateFastIfNotDoneAndBind instead of generateFastAndBind in complex project and heavy computation when generating AST

Bidirectional binding

To have a complete vision of the meta-models described above, we give the corresponding figures of each meta-model FAST and FAMIX:

Once Carrefour has been called and the binding is done, we will have the first links between the meta-models as follows:

From FAST to Famix

As an example, for the condition level variable (a>b) in FAST we would want its correspondence in FAMIX. To do this, we send the #famixVariable message to the FASTJavaVariableExpression object and we get as returned value the corresponding FAMIX variable.

From Famix to FAST

Now we go in the opposite direction, we will access all the matches of the FAMIX variable a in the FAST meta-model.

To do so, we use the #fastAccesses message as in the figure:

Carrefour also provides the #fastDeclaration API to get where a Famix variable has been declared at the FAST meta-model level.

Conclusion

In conclusion, Carrefour allows us to go back and forth between FAST and FAMIX meta-models. The example used in the blog post is not complex but allowed us to see how to navigate between the two meta-models. In large projects, where there are more initialization, invocation, and relationships between entities Carrefour is crucial to perform deep analysis 💪.

Tags:

• FAST

Load FAST Pharo/Java model

Jun 22, 2022

Ahmed Zaki BENNECER

Working on data flow analysis project for my internship

Introduction

When we are interested in the migration/modernization of projects we are using models of the project and their meta-models. Moose revolves around a powerful Famix meta-model that allows us to do several operations. For instance, previous posts present how to analyze and query a model, visualize a model with plantUML, or create a model, etc.

FAST is a meta-model that helps us understand source code in a less abstract way. Indeed, FAST is based on AST (Abstract Syntax Tree) which is close to the source code. And as the devil is in the details, FAST contains interesting elements when analyzing programs (for example some specific expression or statement), and effectively this is what makes the difference between FAST and Famix. (Consult this overview about the FAST model).

In this blog post we will explain how to load FAST Java and generate a FAST model of Java code. For this we will take ArgoUML, an open-source Java project, as an example.

Overview & Prerequisites

First of all, we have to understand from where we are going to start and where we are going to end up. As already mentioned, we will take the AgroUML’s java code and the goal is to generate the corresponding AST and do analyses on it. To do this, 3 steps are necessary to have the AST as illustrated in the figure below:

1. Parse Java to build a Famix model
2. Load the model into Moose
3. Generate the AST

Before starting, we must download the source code and the Famix model of the ArgoUML project, step 1 of the diagram above (follow this blog for more details).

Configuration

Now, we will import the Famix model from the ArgoUML-0-34.json file in the Models Browser. Then, we should know that the FAST meta-model is specific to a gien programming language, i.e for Pharo code we need FAST for Pharo, for X language code we need the FAST meta-model for the X language. Right now, there are two FAST meta-models: FAST Java and FAST Pharo.

In the following, we will generate the AST of a class (or method) for Pharo/Java code in three different ways: directly from some source code, from a method in Pharo, or from a Famix entity.

FAST Installation

To install FAST Java you can run the following script on Moose Playground:

Metacello new
    githubUser: 'moosetechnology' project: 'FAST-JAVA' commitish: 'v3' path: 'src';
    baseline: 'FASTJava';
    load: 'all'

To install FAST Pharo use the following script:

Metacello new
baseline: 'FASTPharo';
repository: 'github://moosetechnology/FAST-Pharo:v2/src';
load: 'importer'.

Java code as string

In this case, we will use a specialized importer “FAST-Java importer” to import the AST from a method source code. The complete code of the method to import is between single quote (i.e. a Pharo string) in the following code:

JavaSmaCCProgramNodeImporterVisitor new
  parseCodeMethodString: 'public boolean covidTest(Person person) {
    if(testCovid(person) == "POSITIVE"){
      return true;
    } else {
      return false;
    }
  }'

Pharo class or method

The following script imports the method #collect: of Collection :

FASTSmalltalkImporterVisitor new
  runWithSource: (Collection >> #collect:) sourceCode

Generate FastJava model

In this section, we will not proceed as above. Instead, we start from a class/method of the Famix Java model and we will load its FAST representation.

We will add the model to the Playground

We got this:

argoUML034 := MooseModel root at: 1.

We pick any model class from the model:

class := argoUML034 allModelClasses anyOne.

And finally we generate the AST using generateFastJava:

class generateFastJava

Navigating Through AST

One nice way to explore a FAST model is to use the source code and the tree extensions of the inspector. It allows one to navigate in a FAST model and see the code corresponding to each node.

To use it, we start from the Java model loaded above. Then, we select a model method entity. On the right-hand pane of the inspector, select the Tree tab, on the left-hand pane, select the source code extension. The source code is highlighted and the area selected corresponds to the entity selected in the right-hand panel. ( from FAST-Pharo article )

Conclusion

In this post, we saw how to load the AST of a Pharo/Java model using FAST. The FAST model is useful when we need to understand more details about our model (for example identifiers, expression statements .. etc) which are not provided by Famix.

Tags:

• FAST

How to build a toolbar with Spec

May 9, 2022

Clotilde Toullec

Research engineer

How to build a toobar with Spec

Let’s build a simple presenter showing a counter. To manage this counter, we will build a toolbar which buttons increase, decrease or reset the value of this counter.

Our presenter is implemented in CounterPresenter, a subclass of SpPresenter. It will define 3 methods to manage the counter value: #increaseCounter, #decreaseCounter and #resetCounter. We will not consider the building of the presenter itself but we will focus on the toolbar.

Simple version: build a toolbar manually

Spec provides an API to build toolbars and add dedicated buttons in it. We will use it in the presenter initialization method: #initializePresenters, overriden from SpPresenter, to instantiate a toolbar:

toolbar := self newToolbar

Then, we manually build toolbar buttons and add them into the toolbar:

    toolbar
        add: (SpToolbarButtonPresenter new
            label: 'Increase';
            icon: (self iconNamed: #smallAdd);
            action: [ self increaseCounter ]);
        add: (SpToolbarButtonPresenter new
            label: 'Decrease';
            icon: (self iconNamed: #remotesManagerRemoveRemote);
            action: [ self decreaseCounter ]);
        add: (SpToolbarButtonPresenter new
            label: 'Reset';
            icon: (self iconNamed: #smallUpdate);
            action: [ self resetCounter ]).

We also need to add the toolbar to the presenter layout. Since Pharo 10, we can do this instance side:

    intializeLayout

        self layout: SpBoxLayout newTopToBottom
            add: toolbar height: self class toolbarHeight;
            "... other widgets ...";
            yourself

This version works perfectly. However, the definition of the buttons, their actions, labels and icons are only defined locally and are not easy to reuse. We will extract this behavior to Commands and use them to build our toolbar.

Reify actions: Commands

Extract our buttons behavior to Commands

We use Commander, the implementation of the Command pattern in Pharo. Let’s create 3 subclasses of CmCommand, one for each of our buttons. These classses define an instance variable #context. In this case, it will be our presenter.

Each class should define the #execute method according to its expected behavior:

execute

    self context increaseCounter

Our commands should also implement the class methods #defaultTitle, and #defaultIconName.

defaultTitle

    ^ 'Increase'

defaultIconName

    ^ #smallAdd

Use commands in Spec

To be used in Spec, our commands are decorated as SpCommands via the method #asSpecCommand. We override this method to correctly use the icon name, as follows:

asSpecCommand

    ^ super asSpecCommand
        iconName: self class defaultIconName;
        yourself

Convert commands to toolbar buttons

SpCommand provides an API to be converted as buttons in spec. A first way to do it is to directly add them in the toolbar. Here is a second version of the #initializePresenters method:

    toolbar
        add: (IncreaseCounterCommand forSpecContext: self) buildPresenter;
        add: (DecreaseCounterCommand forSpecContext: self) buildPresenter;
        add: (ResetCounterCommand forSpecContext: self) buildPresenter.

Here, we give the presenter as context for the commands before building each button.

At this step, we have a functional toolbar using built with commands. We can still improve this by using commands groups

Command groups

Spec presenters can define commands groups to be used in toolbars and menus via the class method #buildCommandsGroupWith:forRoot:. We implement it in our presenter, on class side:

buildCommandsGroupWith: aPresenter forRoot: aRootCommandGroup

    aRootCommandGroup
        register: (IncreaseCounterCommand forSpecContext: aPresenter);
        register: (DecreaseCounterCommand forSpecContext: aPresenter);
        register: (ResetCounterCommand forSpecContext: aPresenter)

To get this command group, SpPresenter implements #rootCommandsGroup. This method collects the commands defined in #buildCommandsGroupWith:forRoot: and set the current presenter as their context. We call it in the #initializePresenters method.

toolbar fillWith: self rootCommandsGroup.

⚠️ Be careful, #fillWith: will remove all items already present in the toolbar. In this code snippet, aButton will not be in the toolbar:

toolbar
  add: aButton; "Will be removed by next line"
  fillWith: aCommandGroup.

Use window toolbar

Instead of defining a toolbar as a subpresenter, it is a good practice to define the toolbar in the window directly. We remove the toolbar instance variable and all the related code in #initializePresenters and #initializeLayout. We then override #initializeWindow: from SpPresenter.

initializeWindow: aMiWindowPresenter

    super initializeWindow: aMiWindowPresenter.
    aMiWindowPresenter toolbar:
        (SpToolbarPresenter new fillWith: self rootCommandsGroup)

Conclusion

Building toolbars in Spec can be done manually. However, by using commands, we separate responsibilities and we can re-use, extend and test these commands. The commands our presenter builds can be used not only in toolbars, but also in menus in a similar manner.

Tags:

• infrastructure

Introducing the new Queries Browser

Oct 10, 2021

Sebastian Jordan

Moose developer

What is a Famix Query?

Let’s say that you want to know which classes of your Moose model are stub. That means: which classes are not defined in your Moose model but are used by some of your defined classes. Those classes are part of your model, although they are not part of your code. Checking those stub classes is easy. You only have to create a Boolean query (assuming that your model contains only classes) with the property isStub. Like:

FQBooleanQuery property: #isStub

However, creating queries programmatically can be a tedious task. Of course, not all the queries are as frivolous as the one in the example. For queries with lots of children, the code is not easy to understand at first sight.

Queries Browser

The new Queries Browser was developed to create queries in a more visual way. This is a Moose tool that allows one to create and manipulate queries without the need of knowing the FamixQueries syntax or how to instantiate them. It has a friendly and intuitive user interface.

If you want to know:

1. What are all the classes that are not stub.
2. What are the entities with incoming references and inheritances.
3. What are the entities that have more than 50 lines of code.
4. Finally, what is the intersection of those three queries.

This is an easy task for the Queries Browser. First, we need to create a type query that filters all the entities except the classes. To do that, we select Type Query in the queries browser and then select type “Classes”.

Then, we create a child query from the Type Query. This child is going to be a Boolean Query that has the property isStub.

Now, we create a Complement Query, a.k.a Negation Query, and choose the Boolean Query to be the query to be negated.

Now we have the first task completed: All the classes that are not stub.

For the second task, we need to create another query, a Navigation Query, select the Incoming direction, and only select the associations Reference and Inheritance.

The third one is also simple. We only need to create a Numeric Query, select the property number of lines of code, the operator >, and put the value 50.

Now, our Queries Browser looks like this:

For the final task, we need to create an And Query, a.k.a Intersection Query, click on the ”+” button to add a new query to intersect, and select the three previous queries that we created.

If we wanted to the the above queries programmatically, the code would have look like this:

(FQComplementQuery queryToNegate:
   (FQTypeQuery types: { FamixStClass })
   --> (FQBooleanQuery property: #isStub))
& (FQNavigationQuery incoming associations: {
     FamixStReference.
     FamixStInheritance }) & (FQNumericQuery
   property: #numberOfLinesOfCodeWithMoreThanOneCharacter
   comparator: #>
   valueToCompare: 50)

The code is also shown in the “Current query code tab”:

Saving and loading the queries

As you may already noticed, there are two button for saving and loading the queries. The save button saves all queries that are currently present on the queries browser as a STON file. The queries will be saved inside a folder in the same location as the image. The path is determinated in MiSaveQueriesCommand class>>#path.

When loading the queries, the saved queries will be put after the queries that are present in the browser, if any. For example, if we save the queries that we have created above.

Then, with an empty browser, we create a new query and then loading the file we get:

Conclusion

The new Queries Browser can simplify how the Famix Queries are created and make it more visual and understandable. Even if the example in this post is not complex, it made our tasks easier. Analyzing models with real-life examples can lead to very nested queries. Create those queries programmatically can be very tedious and error-prone. The Queries Browser is here to help us in those cases. 😄

Tags:

• browser

Newer posts

Older posts