Documentation

DroppedNeedle replaces Lidarr with a built-in library and download engine. There is no Lidarr, no coexistence, and no toggle. This page covers what the engine does and how each stage works.

Legality boundary

The engine communicates only with a user-supplied slskd instance over its local HTTP API. It has no Soulseek protocol code, only an HTTP client for slskd. It never joins or distributes on the Soulseek/P2P network. It issues searches and download requests to slskd and imports the results. You supply, run, and are responsible for slskd and its shared folders. See slskd Setup.

Architecture

The backend is layered, and layers are never skipped:

Routes (api/v1/routes/)            thin HTTP wrappers, auth-checked
  -> Services (services/native/)   scanner, orchestrator, matcher, file processor
    -> Repositories                external I/O: slskd, MusicBrainz
      -> Infrastructure            tagger (mutagen), fingerprinter (fpcalc/AcoustID),
                                   persistence (SQLite WAL), HTTP, SSE

Persistence is two SQLite (WAL) stores: LibraryDB (scanned and imported files, album metadata) and DownloadStore (download tasks, search jobs, quarantine). Progress is streamed to the frontend over SSE.

Library scanner

LibraryScanner walks the configured library paths a folder at a time. For each file it runs the identification pipeline below. The scan is resumable from a progress ledger, cooperatively cancellable, and incremental: a file whose mtime and size are unchanged since the last scan is skipped. It ends with a soft-delete reconcile and a canonical-artist pass.

Identification tiers

Identification stops at the first confident match. There are four tiers:

Tier 1 (MBID in tags): the file already carries a MusicBrainz release-group and recording ID. Accepted at confidence 1.0, no network call.

Tier 2 (fuzzy text match): artist, album, and title are matched against MusicBrainz using rapidfuzz.token_set_ratio. Accepted at confidence >= 0.85.

Tier 3 (AcoustID fingerprint): fpcalc fingerprints the audio and AcoustID resolves a recording to a release group. Accepted at score >= 0.70. fpcalc is bundled in the image via libchromaprint-tools. An AcoustID API key is optional but enables this tier.

Tier 4 (manual review): nothing met the confidence threshold. The file goes into a queue for an admin to resolve: accept the top candidate, supply an MBID, or reject.

Performance

Library reads aggregate from library_files (GROUP BY release_group_mbid) and are sub-second for a 10k-album library on SQLite WAL. An initial scan of a 10k-album library takes roughly 50 minutes when around 30% of files need a MusicBrainz lookup, because the MB client is rate-limited to 1 request/second. Subsequent scans are incremental and far faster.

Download sources

The engine acquires from two sources: your own slskd over Soulseek, and Usenet through SABnzbd with Newznab indexers. Enable either or both. When both are on, a source-priority setting decides which is tried first for a given request. Every download, wherever it comes from, runs through the same scoring, verification, quarantine, and import described below. See slskd Setup and Usenet Setup.

Download pipeline

DownloadOrchestrator owns the full request lifecycle: search, score, auto-pick, enqueue, poll, process, import, notify.

Steps

  1. A request creates a download_tasks row and dispatches the orchestrator.
  2. The orchestrator searches via the configured download client (slskd, SABnzbd, or both by source priority) and ranks the results with AlbumPreflightScorer. Per-track requests use TrackMatcher.
  3. The top candidate is auto-picked if score >= 0.70, parked in a review queue if score falls between 0.50 and 0.70, or the task fails if no candidate scores >= 0.50.
  4. A DownloadManifest (source peer, expected filenames, and durations) is written to staging/{task_id}/manifest.json, then the files are enqueued in slskd.
  5. The orchestrator polls slskd until the transfer completes.
  6. FileProcessor processes each file independently; one bad file never aborts the rest of an album.
  7. Status resolves to completed (all imported), partial (some failed), or failed.

Preflight scoring

Candidates are grouped by (peer, folder). Each group gets a coherence score, then a final score:

coherence = 0.40 * (file_count / expected_track_count)   # completeness
          + 0.20 * dir_name_similarity                    # folder vs "artist album year"
          + 0.15 * format_consistency                     # all FLAC = 1.0
          + 0.15 * bitrate_consistency                    # low stddev
          + 0.10 * no_junk_bonus                          # not "Various/Unknown"

final = 0.50 * coherence
      + 0.30 * avg_file_confidence
      + 0.10 * upload_speed_signal
      + 0.10 * free_slot_bonus

Per-file confidence weights: title (0.55), artist-from-path (0.20), duration tolerance (0.25). A version mismatch (remix, live, or acoustic matched against the original, or vice versa) is penalised by x0.3. A quality gate drops candidates outside the configured codec and quality tier range. Ranking prefers the highest quality tier absolutely: any acceptable FLAC beats any MP3.

Verification, import, and quarantine

FileProcessor resolves each finished file in slskd’s downloads directory and processes it independently.

Verification: tags must be readable. Duration must be within tolerance of the manifest’s expectation. When AcoustID verification is enabled, the fingerprint’s release group must match.

Import: a file that passes verification has MBID tags written, a target path computed from the naming template, and is moved into the library with an atomic os.rename. A cross-filesystem case (which should not happen with a correctly configured bind-mount) falls back to copy-then-remove. A library_files row is inserted.

Quarantine: a file that fails verification (verify_failed, corrupt, fingerprint_mismatch, duration_mismatch) records a download_quarantine row keyed by (client_id, peer, filename, release_group). The scorer excludes that (peer, filename) from every future ranking, so a known-bad source is never re-picked.

Environment faults (a missing or inaccessible downloads mount) are not quarantined. They are not the source’s fault. The file fails with a sanitised reason instead.

Held-import review

A candidate scored between 0.50 and 0.70 isn’t confident enough to auto-accept, so its download is parked for review rather than imported. An admin opens the held-import queue, previews the audio, and either accepts it (optionally supplying a MusicBrainz ID) or rejects it. Accepting runs the file through the same verification and import as an auto-picked download. Separately, when a download finishes in your client but the engine can’t locate the file, a manual reimport from the Downloads page points it at the file to finish the job.

Upgrades and storage

Ranking already prefers the highest quality tier during preflight scoring. On top of that, when a strictly better copy of an album you own turns up, the engine replaces the files on import and moves the originals to a recycle bin; an opt-in background scan looks for these upgrades on a schedule. Album edition pinning locks a request to one release, filling missing tracks from that edition and upgrading files as better copies surface. A global storage cap and per-user request and storage quotas bound how much the library can grow.

Wanted watcher

Requests that fail or import only partially are handed to a background watcher. It re-searches them on an age-based cadence, and when a verified match turns up it downloads and imports without prompting. The Wanted tab lists what the watcher is tracking, with per-item stop and resume controls and a way to mark new finds as seen.

Naming template

Imported files are placed using a configurable template. Available variables:

{artist} {album} {albumartist} {year} {track:02d} {title} {ext}
{disc} {genre} {medium} {musicbrainz_id} {artist_mbid}

Default template:

{albumartist}/{album} ({year})/{disc:02d}{track:02d} {title}.{ext}

The template applies to downloaded imports only. The scanner never renames files it discovers. Changing the template does not retroactively reorganise the library.

Cover art

Album covers resolve on demand through AlbumCoverFetcher. The fetch order is:

  1. AudioDB
  2. Local sources (an existing library file or Jellyfin)
  3. MusicBrainz Cover Art Archive
  4. Best-release fallback

First success wins.

Download client protocol

The engine speaks a DownloadClientProtocol, not slskd or SABnzbd directly. The interface defines: client_name, is_configured, health_check, search_album, search_track, enqueue, get_status, cancel, get_file_path. Everything client-specific lives inside that client’s repository. Everything else (library layout, MusicBrainz identification, the atomic move, tag writing, persistence, quarantine, scoring) lives outside it. slskd and SABnzbd are both implemented behind this protocol, and adding a further client requires no changes to services/native/.

Auth posture

  • Library catalogue reads and download status: any authenticated user.
  • User-scoped download tasks and searches: owner or admin, with ownership checked in the service layer.
  • Scan control, download-client configuration, quarantine, and tag editing: admin only.
  • SSE endpoints require auth on subscribe and are ownership-scoped for download streams.
  • API keys are masked on settings reads and never appear in logs.