Post

Ongoing ArKanjo Contributions

Posted Updated
By Raffael Raiél Trindade
4 min read
Ongoing ArKanjo Contributions

FLOSS Discipline 2nd Phase

The Linux Kernel Development posts were relative to the first phase of the FLOSS discipline, as stated in the first diary.

After finishing the contribution on vcnl4000.c driver, it was requested that the studentes contributed to another FLOSS project related to the Linux ecossytem. At first, we chose MLHeritage for processing mailing lists data. Although, the distance imposed by Kim’s internship led to organization issues and the complexity embeded on the issues was far too high for us to contribute in a fast fashion.

Therefore, we moved to the ArKanjo project, devloped by Gui Ivo aiming to detect code duplication using different tools.

There was a marged contribution, a waiting PR and two issues openned that we would like to solve, also a discussion on real-world usage.

Cluster Bug

This already merged contributtion revolved around the arkanjo explore -c ... command.

The explore command shows the duplicated functions and number of lines, whilhist the -c flag clusters the functions and lines in order to infer what group of functions share duplications.

When running this command, all number of duplicated lines shown were -1, the default value on the struct on src/commands/explorer/similarity_explorer_entry.hpp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

struct SimilarityExplorerEntry {
  std::string path_a;
  std::string path_b;
  std::string dir_a;
  std::string dir_b;
  std::string filename_a;
  std::string filename_b;
  std::string func_a;
  std::string func_b;
  int start_a;
  int start_b;
  int end_a;
  int end_b;
  int duplicated_lines{-1};
};

The solution was on the argument passing on the src/commandsexplorersimilarity_explorer.cpp on the explorer_clusters() function.

At first, the code was:

1
2
3
4
5
6
7
8
9
10

std::vector<SimilarityExplorerEntry> entries{};
for (const auto& path : info.paths) {
    entries.push_back({
        path.format_path_message_in_pair(), "", 
        path.build_relative_path().parent_path().string(), "",
        path.build_relative_path().filename().string(), "",
        path.build_function_name(), "",
        find_number_lines(path)
    });
}

and let us note that there are fewer parameters than the requested by SimilarityExplorerEntry struct. Also, the function find_number_lines, givne by:

1
2
3
4
5

int SimilarityExplorer::find_number_lines(const Path& path1) {
    Function function(path1);
    function.load();
    return function.number_of_lines();
}

is a wrapper to get the number of lines in a given function.

To solve the given bug, we decided to process all the information lost, also assigning surrogate values to the non-existing “b function” in the -c option processing.

The final form was:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

std::vector<SimilarityExplorerEntry> entries{};
for (const auto& path : info.paths) {
    Function function(path);
    function.load();
    entries.push_back({
        path.format_path_message_in_pair(), "", 
        path.build_relative_path().parent_path().string(), "",
        path.build_relative_path().filename().string(), "",
        path.build_function_name(), "",
        function.get_scope_function_in_file()[0], 0,
        function.get_scope_function_in_file()[2], 0,
        function.number_of_lines()
    });
}

and so, by removing the wrapper, we could unstate it from the similarity_explorer.cpp and the header similarity_explorer.hpp.

Therefore, we successfully transformed this pattern:

1
2
3
4
5
6
7
8
9
10

Cluster #5 (Files: 8, Pairs: 28, Lines: 504, Score: 209.400000)
---------------------
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/code.c::find_min, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_equal_code/code.c::find_max, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_equal_code/code.c::find_min, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/source.c::find_min, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/source.c::find_max, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_equal_code/source.c::find_max, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_equal_code/source.c::find_min, TOTAL NUMBER LINES IN FUNCTION: -1
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/code.c::find_max, TOTAL NUMBER LINES IN FUNCTION: -1

into this:

1
2
3
4
5
6
7
8
9
10

Cluster #5 (Files: 8, Pairs: 28, Lines: 504, Score: 209.400000)
---------------------
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/code.c::find_min, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_equal_code/code.c::find_max, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_equal_code/code.c::find_min, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/source.c::find_min, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/source.c::find_max, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_equal_code/source.c::find_max, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_equal_code/source.c::find_min, TOTAL NUMBER LINES IN FUNCTION: 9
Function find: tests/e2e/codebase/test_multiple_file_shuffled_functions/code.c::find_max, TOTAL NUMBER LINES IN FUNCTION: 9

This contribution was merged right away since it fixed the problme straighforwardly.
The commit can be found on https://github.com/arkanjo-tool/arkanjo/commit/6d290a88f36ddd65d7c71d34e166d5cbfa72e793.

Other Contributions

We intended to run ArKanjo on Aurora, the BCC’s undergraduate progression tracking system.

While trying to do so, we ran into some problems:

  • libgit2 system/local conflict;
  • PHP tree-sitter unexistent;
  • ”~” and “.”, “..” not trated.

Libgit2

For the first, Kim fixed the cmake/Targets.c by changing

1

set(ARKANJO_LIBGIT2_LIBS git2)

to

1

set(ARKANJO_LIBGIT2_LIBS libgit2::libgit2package)

and

1

${libgit2_SOURCE_DIR}/include

to

1

$<$<NOT:$<BOOL:${ARKANJO_LIBGIT2_SYSTEM}>>:${libgit2_SOURCE_DIR}/include>

With this, the conflicts betrween local and global installs were removed (tested by changing DARKANJO_LIBGIT2_SYSTEM values duting building).

This change is waiting to be reviewed and merged.

Openned Issues

We also oppened issues for the second and third problems encountered. At the moment, I am trying to make my solution to the third to work, but some problems are occurring on the .cache file access when using “~” alias to $HOME. They can be accessed on:

Aurora Analysis

After fixing #2, we expect to analyse duplication on Auroras source code and, maybe, try to solve them in near future.

floss
This post is licensed under CC BY 4.0 by the author.