semdup
github.com
- Windows
- macOS
- Linux
- Self-hosted
What semdup is
semdup is an open-source repository scanner that finds code with similar meaning, even when token sequences differ. It is designed for Linux, macOS, Windows, and self-hosted use, and can help identify AI-generated duplication or stop coding agents from rebuilding existing code. Its default CodeRankEmbed model runs on CPU or CUDA locally, while an optional Amazon Bedrock backend uses Titan Text Embeddings V2 and requires AWS credentials plus bedrock:InvokeModel permission. Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, C, and C++ are listed as supported languages, with weaker results possible for heavily macro-based C and C++. Similarity, warning, error, and duplicate-count thresholds are configurable. GitHub Actions integration can check pull requests and pushes, while diff mode compares changed code with a Git base reference. SQLite stores indexes and cached embeddings.
Who it suits
semdup suits development teams reviewing repositories for semantically duplicated code, including AI-assisted projects. It also fits teams wanting local inference, self-hosting, or CI checks.
Good to know, and what to watch
- Semantic matching beyond token sequences
- Runs locally on CPU or CUDA
- Configurable similarity and warning thresholds
- GitHub Actions workflow and action
- Supports multiple programming languages
- Sparse search can miss exact-search matches
- Fuzzy matching can produce false positives
- Heavily macro-based C/C++ may perform less well
- Project may contain bugs
The verdict
semdup combines semantic repository scanning, configurable matching, local inference, and CI integration. Its approximate sparse mode and stated early-stage limitations should be considered when choosing search settings.
What the maker publishes
- Bedrock integration
- An optional Amazon Bedrock backend uses Titan Text Embeddings V2 through the AWS SDK credential chain and requires bedrock:InvokeModel permission. source
- Caveat
- The project says it is still young, may have bugs, and can produce false positives because it uses fuzzy search. source
- Code review
- The CLI includes a diff mode that compares changed code against a Git base reference, and the project provides a GitHub Actions workflow and action. source
- Configuration
- Users can configure similarity thresholds, warning and error thresholds, and the minimum number of duplicates in a cluster. source
- Download integrity
- The first initialization, refresh, or scan downloads the model into a user cache and verifies files against compiled BLAKE3 hashes before loading them. source
- Index limitation
- Sparse search is approximate and can miss pairs found by exact search, while exact search compares every pair and avoids dropping matches. source
- Known limitation
- The maker says the tool is still young, may contain bugs, and can produce false positives because it uses fuzzy search. source
- Languages
- The project lists Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, C, and C++ as supported languages, with a caveat that heavily macro-based C/C++ may perform less well. source
- Languages
- It supports Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, and C/C++, with a note that heavily macro-based C/C++ may perform less well. source
- License
- The project is available under either the MIT or Apache-2.0 license, at the user's option. source
- License
- The project is available under either the MIT or Apache-2.0 license. source
- Local inference
- The default model runs locally on CPU or CUDA, and the maker says semdup keeps everything local unless an external backend is chosen. source
- Local inference
- The default model runs on the user’s machine using CPU or CUDA. source
- Maintainer support
- The maker welcomes contributions to the project. source
- Matching controls
- Users can tune similarity thresholds, configure separate warning and error thresholds, and require matches to appear a chosen number of times. source
- Model choice
- The default embedding model is nomic-ai/CodeRankEmbed, and users can configure a compatible model or use a Python sidecar for other backends. source
- Optional backend
- An optional Amazon Bedrock backend uses Titan embeddings and the AWS SDK credential chain. source
- Purpose
- semdup is a fuzzy-search tool that detects near-duplicate code by comparing semantic meaning rather than token sequences. source
- Purpose
- Semdup detects near-duplicate code by comparing semantic meanings rather than token sequences. source
- Search limits
- Sparse search is approximate and can miss matches that exact search would find; automatic indexing switches to sparse search above 20,000 scannable units. source
- Storage
- Semdup stores its index and cached embeddings in SQLite, keyed by content hash and model ID. source
- Use case
- The maker describes it as useful for catching AI-generated code duplication and preventing agents from reimplementing existing code. source
- Workflow integration
- The project provides a GitHub Actions workflow and action that can run semdup checks on pull requests and pushes. source
Checked against the maker's pages on 9 Oct 2026.