[Fix] 자기소개서 채점 프롬프트 및 서버 검증 강화#111
Conversation
- 채점 프롬프트에 jobFit, impact, completeness 세부 평가 기준 반영 - fabricated status를 단순 근거 부족이 아닌 명백한 모순/비현실적 과장으로 한정 - reason, improvement, sentence 작성 규칙을 프롬프트에 명확히 추가 - LLM score 대신 jobFit/impact/completeness 가중합으로 총점 계산 - 하위 점수 0~100 범위 검증 추가 - invalid status 및 missing 분석 항목 제외 처리 - sentence 원문 포함 여부, 중복 제거, 문항당 최대 3개 제한 강화 - improvement 지시문 표현 제거 로직 보강 - 관련 AnalysisService 테스트 보완
|
Warning Review limit reached
Next review available in: 21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThe AnalysisAiClient prompt logic was refactored into reusable constants with stricter output-validation rules. AnalysisService now validates LLM-returned scores, computes a weighted overall score, and applies stricter sentence/status parsing and deduplication when building question analyses. AnalysisServiceTest expectations and coverage were updated accordingly. ChangesAnalysis prompt and scoring update
Estimated code review effort: 3 (Moderate) | ~30 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java (1)
357-374: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winBanned-phrase list duplicated between prompt text and Java code.
The instruction-like detection logic (regex +
.contains()checks) mirrors the금지 표현list defined inAnalysisAiClient'sSTATUS_AND_WRITING_RULES(Line 149:추가하세요, 보완하세요, 수정해주세요, 수정하세요, 작성해주세요, 작성하세요, 필요합니다, 해야 합니다, 해주세요, 명확히 해야). Today both lists are in sync, but they live in separate files with no shared reference, so a future addition to one (e.g., updating the prompt's banned phrases) can silently drift from the other, letting non-compliantimprovementtext through or over-filtering valid ones.Consider extracting the canonical phrase list into a single shared constant that both the prompt text and the Java filter derive from.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java` around lines 357 - 374, The banned-phrase detection in AnalysisService is duplicated from the STATUS_AND_WRITING_RULES prompt in AnalysisAiClient, which risks the two lists drifting over time. Extract the canonical “금지 표현” phrases into a shared constant or helper that both AnalysisService’s improvement filter and AnalysisAiClient’s prompt generation use, and update the regex/.contains() checks to derive from that single source.src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java (1)
32-32: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUnused
scorefield still required in schema despite server discarding it.
OUTPUT_SCHEMA(Line 32) and the self-verification checklist (Line 250) still instruct the model to producescore, butAnalysisService.analyze(...)now computes the persisted score entirely fromjobFit/impact/completenessweights and never readsllmResponse.score(). SinceAnalysisLlmResponseis bound via.text(AnalysisLlmResponse.class), the field is still part of the derived JSON schema, so the model keeps spending output tokens generating a value nobody uses.Consider dropping
scorefrom the schema/record (or explicitly documenting why it's intentionally retained, e.g. as a self-consistency nudge for the model) to avoid confusing future maintainers and to save tokens.Also applies to: 242-254
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java` at line 32, Remove the unused score requirement from the LLM contract in AnalysisAiClient: update OUTPUT_SCHEMA and the self-verification checklist so AnalysisLlmResponse no longer asks the model to emit score, since AnalysisService.analyze(...) ignores llmResponse.score() and computes the persisted score from jobFit, impact, and completeness instead. If score is intentionally kept for model guidance, document that explicitly in the AnalysisLlmResponse schema and checklist; otherwise delete the field from the record/schema to avoid wasted tokens and confusion.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java`:
- Line 32: Remove the unused score requirement from the LLM contract in
AnalysisAiClient: update OUTPUT_SCHEMA and the self-verification checklist so
AnalysisLlmResponse no longer asks the model to emit score, since
AnalysisService.analyze(...) ignores llmResponse.score() and computes the
persisted score from jobFit, impact, and completeness instead. If score is
intentionally kept for model guidance, document that explicitly in the
AnalysisLlmResponse schema and checklist; otherwise delete the field from the
record/schema to avoid wasted tokens and confusion.
In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.java`:
- Around line 357-374: The banned-phrase detection in AnalysisService is
duplicated from the STATUS_AND_WRITING_RULES prompt in AnalysisAiClient, which
risks the two lists drifting over time. Extract the canonical “금지 표현” phrases
into a shared constant or helper that both AnalysisService’s improvement filter
and AnalysisAiClient’s prompt generation use, and update the regex/.contains()
checks to derive from that single source.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 93eda773-a55c-46a9-98f8-282d63139a9a
📒 Files selected for processing (3)
src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.javasrc/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisService.javasrc/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java
- LLM 응답 DTO와 프롬프트 스키마에서 미사용 score 필드 제거 - 서버 총점 계산 흐름에 맞춰 self-verification checklist 정리 - improvement 금지 표현 목록을 공용 helper로 분리 - 프롬프트와 서버 검증 로직이 동일한 금지 표현 목록을 참조하도록 변경 - AnalysisService 테스트 fixture를 변경된 LLM 응답 DTO에 맞게 수정
1 participant
✨ 어떤 이유로 PR를 하셨나요?
📋 세부 내용 - 왜 해당 PR이 필요한지 작업 내용을 자세하게 설명해주세요
📸 작업 화면 스크린샷
🚨 관련 이슈 번호 [#91 ]
Summary by CodeRabbit