Skip to content

Add ocr service#8

Open
SahilBheke25 wants to merge 2 commits into
stagingfrom
feat/ocr-service
Open

Add ocr service#8
SahilBheke25 wants to merge 2 commits into
stagingfrom
feat/ocr-service

Conversation

@SahilBheke25

Copy link
Copy Markdown

No description provided.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new standalone OCR/PDF text-extraction module under OCR/, designed to be reusable as both a CLI tool and an importable package with a pluggable OCR-engine interface (currently Surya OCR).

Changes:

  • Introduces the extractor/ package with a generic OCR result model, load/format pipeline, and engine registry + Surya engine adapter.
  • Adds a CLI entry point (OCR/main.py) and extensive module documentation (OCR/README.md).
  • Adds module dependencies (OCR/requirements.txt) and updates repo .gitignore.

Reviewed changes

Copilot reviewed 11 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.gitignore Adds an ignore entry for macOS metadata files (currently with incorrect casing).
OCR/README.md Documents goals, architecture, usage, output formats, and extension points for the OCR module.
OCR/main.py Provides a CLI wrapper around Extractor.run() with --engine selection.
OCR/requirements.txt Declares OCR module dependencies (currently unpinned).
OCR/extractor/__init__.py Exposes the public API (Extractor, defaults, ExtractionResult).
OCR/extractor/pipeline.py Implements the high-level extraction pipeline and file output naming.
OCR/extractor/models.py Defines engine-agnostic Block and PageResult dataclasses.
OCR/extractor/loader.py Implements PDF discovery and PDF→image rendering via pypdfium2.
OCR/extractor/formatter.py Builds plain text, JSON, and HTML outputs from engine-agnostic results.
OCR/extractor/engines/__init__.py Adds the engine registry + get_engine() factory.
OCR/extractor/engines/base.py Defines the BaseOCREngine interface contract.
OCR/extractor/engines/surya_engine.py Implements the Surya OCR adapter and lazy backend initialization.
Comments suppressed due to low confidence (2)

OCR/README.md:310

  • The README claims the render output is “capped at ~2048px width”, but loader.py currently renders at a fixed scale and does not enforce any max-width cap. Either implement the cap in code or adjust the docs to avoid promising behavior that isn’t present.
Since sample documents are in Hindi/Marathi/mixed scripts:
- Render scale defaults to produce ~150–200 DPI equivalent (capped at ~2048px width) for legible text — this is the single biggest accuracy lever per Surya's own docs.

OCR/extractor/formatter.py:121

  • The HTML report injects block.html directly into the output. Since OCR output ultimately comes from untrusted PDFs, this can embed unexpected tags (e.g., <script>), which will execute if the report is opened in a browser or served by a web app. Consider sanitizing to an allowlist (tables/paragraphs/etc.) before writing HTML.
        for block in page_result.blocks:
            if block.skipped:
                continue
            parts.append("<div class='block'>")
            parts.append(block.html)
            parts.append("</div>")

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread OCR/extractor/pipeline.py
Comment on lines +131 to +134
try:
rel = pdf_path.relative_to(root).with_suffix("")
return str(rel).replace("/", "__")
except ValueError:
Comment thread OCR/requirements.txt
Comment on lines +1 to +3
surya-ocr
pypdfium2
pillow
Comment thread OCR/README.md Outdated
Comment on lines +103 to +111
def build_html(document_name: str, pages: List[PageResult]) -> str:
"""Build the styled, human-readable HTML report for one document."""
parts = [
"<html><head><meta charset='utf-8'>",
f"<title>{document_name}</title>",
"<style>",
_HTML_STYLE,
"</style></head><body>",
]
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants