Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,25 @@ public record AnalysisLlmResponse(
Integer impact,
Integer completeness,
String feedback,
List<MissingKeywordItem> missingKeywords,
List<QuestionAnalysisItem> questionAnalyses
) {
public AnalysisLlmResponse(
Integer jobFit,
Integer impact,
Integer completeness,
String feedback,
List<QuestionAnalysisItem> questionAnalyses
) {
this(jobFit, impact, completeness, feedback, List.of(), questionAnalyses);
}

public record MissingKeywordItem(
String keyword,
String source
) {
}

public record QuestionAnalysisItem(
Long questionId,
String sentence,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ public record AnalysisResponse(
int impact,
int completeness,
String feedback,
List<MissingKeywordResponse> missingKeywords,
List<AnalysisQuestionResponse> questions
) {
public static AnalysisResponse of(
Analysis analysis,
MockApplyStatus status,
int sequence,
List<MissingKeywordResponse> missingKeywords,
List<AnalysisQuestionResponse> questions
) {
return new AnalysisResponse(
Expand All @@ -33,6 +35,7 @@ public static AnalysisResponse of(
analysis.getImpact(),
analysis.getCompleteness(),
analysis.getFeedback(),
missingKeywords == null ? List.of() : missingKeywords,
questions
);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package com.jobdri.jobdri_api.domain.analysis.dto.response;

public record MissingKeywordResponse(
String keyword,
MissingKeywordSource source
) {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package com.jobdri.jobdri_api.domain.analysis.dto.response;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonValue;

import java.util.Arrays;
import java.util.Optional;

public enum MissingKeywordSource {
QUALIFICATION("qualification"),
PREFERENCE("preference"),
MAIN_TASK("mainTask");

private final String value;

MissingKeywordSource(String value) {
this.value = value;
}

@JsonValue
public String value() {
return value;
}

@JsonCreator
public static MissingKeywordSource fromJson(String value) {
return from(value).orElse(null);
}

public static Optional<MissingKeywordSource> from(String value) {
if (value == null) {
return Optional.empty();
}
return Arrays.stream(values())
.filter(source -> source.value.equals(value.trim()))
.findFirst();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ public class Analysis extends BaseEntity {
@Column(nullable = false, columnDefinition = "TEXT")
private String feedback;

@Builder.Default
@Column(name = "missing_keywords", nullable = false, columnDefinition = "TEXT DEFAULT '[]'")
private String missingKeywordsJson = "[]";

@Builder.Default
@OneToMany(mappedBy = "analysis", cascade = CascadeType.ALL, orphanRemoval = true)
private List<QuestionAnalysis> questionAnalyses = new ArrayList<>();
Expand All @@ -50,6 +54,18 @@ public static Analysis create(
int impact,
int completeness,
String feedback
) {
return create(mockApply, score, jobFit, impact, completeness, feedback, "[]");
}

public static Analysis create(
MockApply mockApply,
int score,
int jobFit,
int impact,
int completeness,
String feedback,
String missingKeywordsJson
) {
return Analysis.builder()
.mockApply(mockApply)
Expand All @@ -58,6 +74,7 @@ public static Analysis create(
.impact(impact)
.completeness(completeness)
.feedback(feedback)
.missingKeywordsJson(missingKeywordsJson == null ? "[]" : missingKeywordsJson)
.build();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ public class AnalysisAiClient {
"impact": 55,
"completeness": 67,
"feedback": "한 줄 피드백",
"missingKeywords": [
{
"keyword": "SQL 활용 경험",
"source": "qualification"
}
],
"questionAnalyses": [
{
"questionId": 1,
Expand All @@ -53,7 +59,8 @@ public class AnalysisAiClient {
4. jobFit, impact, completeness를 각각 평가한다.
5. 감점 금지 조건과 status 오남용 여부를 확인한다.
6. 보완이 필요한 원문 문장을 문항당 최대 3개만 추출한다.
7. 지정된 JSON만 반환한다.
7. JD에는 있지만 자소서에 충분히 드러나지 않은 역량을 missingKeywords로 최대 3개 추출한다.
8. 지정된 JSON만 반환한다.

[jobFit 평가 기준]
JD가 요구하는 역량, 경험, 기술을 자기소개서가 얼마나 증명하는지 평가한다.
Expand Down Expand Up @@ -128,6 +135,18 @@ public class AnalysisAiClient {
- 동일하거나 거의 동일한 문장을 중복 반환하지 않는다.
- start/end index는 출력하지 않는다. 서버가 Java String character index 기준으로 계산한다.
- missing은 원문에 해당 문장이 없을 수 있으므로 sentence를 임의로 만들지 않는다.
- missing은 questionAnalyses에 억지로 넣지 않는다.

[missingKeywords 규칙]
- JD에는 있지만 자소서에 충분히 드러나지 않은 요건이나 역량을 추출한다.
- questionAnalyses와 분리해서 missingKeywords에만 넣는다.
- 최대 3개만 반환한다.
- 누락 키워드가 없으면 null이나 필드 생략이 아니라 빈 배열 []을 반환한다.
- 우선순위는 자격요건(qualification) > 우대사항(preference) > 주요 업무(mainTask)다.
- keyword는 단순 단어보다 짧은 역량 문구 형태로 작성한다.
- 가능하면 JD에 실제 들어간 표현을 유지한다.
- 중복되거나 유사한 keyword는 하나로 묶고, 대표 문구는 자격요건 표현을 우선한다.
- source는 qualification, preference, mainTask 중 하나만 사용한다.

[reason 작성 규칙]
- 사용자가 왜 해당 문장이 보완 대상인지 이해할 수 있게 작성한다.
Expand Down Expand Up @@ -252,10 +271,14 @@ private String buildPrompt(
- questionAnalyses의 questionId는 입력된 questionId 중 하나만 사용한다.
- questionAnalyses의 status는 proven, mentioned, missing, fabricated 중 하나만 사용한다.
- sentence는 answer에 포함된 정확한 substring만 사용한다.
- missing 상태를 questionAnalyses에 넣기 위해 원문에 없는 sentence를 만들지 않는다.
- missingKeywords는 최대 3개이며, 없으면 []로 출력한다.
- missingKeywords의 source는 qualification, preference, mainTask 중 하나만 사용한다.
- improvement가 지시문이 아닌 완성된 한국어 평서문인지 확인한다.
- 원문에 없는 경험, 기술, 도구명, 인원수, 금액, 성과 수치를 만들지 않았는지 확인한다.
- fabricated를 단순 근거 부족에 사용하지 않았는지 확인한다.
- jobFit, impact, completeness는 0~100 정수로 출력한다.
- 총점 score는 서버가 jobFit 50%, impact 30%, completeness 20%로 계산하므로 출력하지 않는다.
Comment on lines +265 to +272

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Confirm no test exercises buildPrompt()/AnalysisAiClient.analyze() with a real prompt build,
# and check for any other unescaped '%' occurrences in text-block templates that feed .formatted().

fd -e java --full-path 'analysis' | xargs rg -n 'AnalysisAiClient' -g '*Test*.java'
rg -n '%[^s%\n]' src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java

Repository: JobDri-Developer/BackEnd

Length of output: 834


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the relevant sections of AnalysisAiClient.java with line numbers.
sed -n '140,180p' src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
printf '\n----\n'
sed -n '225,280p' src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
printf '\n----\n'
# Show the exact line containing the formatted() call, if present nearby.
rg -n '\.formatted\(' src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java

printf '\n----\n'
# Reproduce Java formatter behavior with the exact literal that appears on line 272.
python3 - <<'PY'
s = "총점 score는 서버가 jobFit 50%, impact 30%, completeness 20%로 계산하므로 출력하지 않는다."
print(s)
# Show that Java-style formatting would treat this as invalid by using Python's percent formatter analogously.
# This is just a sanity check for literal percent parsing behavior.
try:
    _ = s % ()
except Exception as e:
    print(type(e).__name__, e)
PY

Repository: JobDri-Developer/BackEnd

Length of output: 4464


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show where buildPrompt() is called relative to try/catch in AnalysisAiClient.analyze().
nl -ba src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java | sed -n '175,205p'

printf '\n----\n'
# Inspect the test to see whether a real prompt build path is exercised.
nl -ba src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java | sed -n '1,180p'

Repository: JobDri-Developer/BackEnd

Length of output: 202


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the analyze() method boundary and the prompt-building helper with portable tools.
sed -n '175,215p' src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java
printf '\n----\n'
sed -n '1,170p' src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java
printf '\n----\n'
# Find whether the test imports or mocks AnalysisAiClient or its dependency chain.
rg -n 'AnalysisAiClient|`@MockBean`|`@Mock`|analyze\(' src/test/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisServiceTest.java

Repository: JobDri-Developer/BackEnd

Length of output: 14048


Escape the percent signs in this formatted prompt. Unescaped % in the outer .formatted(...) string will fail at runtime (50%, 30%, 20% need %%), so analyze() can crash before the OpenAI call is made.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/main/java/com/jobdri/jobdri_api/domain/analysis/service/AnalysisAiClient.java`
around lines 265 - 272, The formatted prompt in AnalysisAiClient.analyze()
contains unescaped percent signs, which will break the outer .formatted(...)
call at runtime. Update the prompt text so the percentage values used for the
server-side score weights are escaped as literal percent signs, and verify the
surrounding prompt string in the analyze() method remains valid after
formatting.

""".formatted(
OUTPUT_SCHEMA,
EVALUATION_CRITERIA,
Expand Down
Loading