compare-codebases

jscpd --compare A B pairs every function of folder A with the function of folder B that does the same job, and lists the functions of each folder that have no counterpart. The two folders may be in one language or in two (Python and TypeScript, Kotlin and Swift, Java and Rust). This skill explains how the comparison works and how to run one well. For porting code with the comparison as the progress measure, use the code-migration skill; for the rest of jscpd, the jscpd skill.

How the comparison works

jscpd finds the functions of JavaScript, TypeScript, JSX, TSX, Vue, Svelte, Astro, Python, Rust, Go, Java, Kotlin, C#, C, C++, PHP, Ruby, Scala and Swift files in both folders. It turns the code of each function into a vector with a code embedding model that runs inside jscpd (CodeRankEmbed by default), so two functions that do the same job point the same way even when their languages, names and structure differ. Functions of one folder are never compared with each other.

Functions pair in two steps:

By code. Two functions pair when each is the other's closest match in the other folder, their cosine similarity reaches the model's threshold (0.4125 across languages and 0.6375 within one language, with CodeRankEmbed), and the similarity stands out from the function's other matches. A function close to the best one also pairs when it reaches a higher bar, so a feature written twice on one side gets two pairs. A pair stays only when it is the best match of at least one of its two functions, so a lookalike from another class does not join a function and its counterpart; a tie goes to the file of the same name. Functions shorter than --min-tokens (30 with --compare) or --min-lines (5) stay out of this step, because a short function resembles too many others.
By name. A function left over pairs with a function of the other folder under the same name, once case, underscores, spaces and punctuation are ignored (encodeBinary, encode_binary, _encode_binary; the test title rounds cents and rounds_cents), when their similarity reaches the medium level (0.5625 across languages with CodeRankEmbed). A name pair skips the closest-match and stand-out checks of the first step, so it needs more than that step's threshold; otherwise every load and init of two codebases would pair. Size does not matter here, so a short port is found. A name pair has to stay within modules the first step linked. A module is the folder right under the deepest folder all files of a side share (notification in android/notification/…), and a file that sits higher than the rest, such as a build script, does not move that folder up. Two modules link when one holds the most of the other's pairs, counted for code and for tests separately. So checkPermissions of one plugin does not pair with its namesake in another.

Each pair gets a level on the scale of the model, because a cosine that is high for one model is low for another: