1. Pick a folder, or paste two snippets
Reads source files (JavaScript, TypeScript, Python, Rust, Go, Java, C, C++, C#, Ruby, PHP, Swift, Scala, shell, CSS). Skips node_modules, .git, target, files over 100 KB and files over 1,000 lines, the same limits jscpd uses.
Or paste two snippets
2. What was copied
Select a row to see both copies. "Copy" is the later file in path order; that is not a claim about which one was written first.
| Tokens | Lines | Copy | Also at |
|---|
How it works
- Tokenize each file: identifiers, numbers, strings and punctuation, with whitespace and comments dropped, so a re-indented or re-commented copy still matches.
- Hash every run of 50 tokens with a rolling hash, and sort the hashes so equal runs sit side by side. Each run is checked token by token, so a hash collision cannot report a false clone.
- Merge neighbouring matches into the longest block, and keep blocks of at least 5 lines on both sides.
This is the method jscpd and PMD's CPD use, with jscpd's defaults. Matching is exact: a copy whose variables were renamed is not found.
Measured, not promised
On 1,054 .mjs files (142,572 lines) from this project's own repository, on a 4-core cloud container:
| Tool | Time | Peak memory | Duplicated lines |
|---|---|---|---|
| jscpd 4.3.0 (Node 22) | 40.1 s | 927 MB | 31.9% |
| dupehound CLI (native, 4 threads) | 0.14 s | 58 MB | 32.7% |
| dupehound in WebAssembly (one thread, Node 22) | 0.23 s | n/a | 32.7% |
The two tools flagged the same lines to within about 6%: dupehound marked 93.5% of the lines jscpd marked, and jscpd 94.6% of dupehound's. The difference is mostly how each splits strings into tokens. On the whole tree (9,458 files, 1.27 million lines) dupehound took 1.2 seconds and 500 MB. jscpd ran out of memory at Node's default heap after 277 seconds; given a 12 GB heap it finished in 337 seconds at a 6.2 GB peak, and the two again flagged the same lines to within about 5%. The method, the scripts and the raw numbers are in the benchmark write-up.
On the command line
git clone --depth 1 --filter=blob:none --sparse https://github.com/duncan-haywood/oss-monorepo cd oss-monorepo && git sparse-checkout set ventures/dupehound cargo install --path ventures/dupehound dupehound src/ # top 20 clones and a summary dupehound --json src/ > clones.json # every clone, machine-readable dupehound --threshold 5 src/ # exit 1 if over 5% duplicated (for CI)
Needs a Rust toolchain (rustup). Respects .gitignore. dupehound --help lists every option.
When to use something else
- jscpd when you need its 150-plus language grammars, HTML or badge reports, git blame on each clone, or its "weak" mode. On a project of a few hundred files its speed is fine.
- PMD CPD when you already run PMD for Java and want one report.
- A clone detector that normalises names (several IDEs and SonarQube do some of this) when the copies you care about had their variables renamed. dupehound will not find those.
Questions
Is my code uploaded?
No. The folder you pick is read by this browser tab and handed to a WebAssembly module loaded from this site. Nothing is sent anywhere; the page makes no network requests after it loads.
How is this different from jscpd?
Same method and defaults, written in Rust. On 1,054 JavaScript files jscpd took 40 seconds and dupehound 0.2 seconds, and they flagged the same lines to within about 6%. jscpd knows far more languages and has richer reports; dupehound has one generic lexer and JSON or text output.
Does it find code that was copied and then renamed?
No. Matching is exact on token text after dropping whitespace and comments. jscpd's default mode behaves the same way.