Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,8 @@ diffscope doctor --base-url http://localhost:11434
|----------|-------------|
| `DIFFSCOPE_BASE_URL` | LLM API base URL (also accepts `OPENAI_BASE_URL`) |
| `DIFFSCOPE_API_KEY` | API key for the LLM endpoint |
| `DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS` | Comma-separated `pull_request` actions that start webhook reviews, such as `opened,synchronize` or `review_requested` |
| `DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS` | Comma-separated GitHub logins whose requested reviews trigger `review_requested` automation, for example `EvalOpsBot` |

#### CLI Flags
| Flag | Description |
Expand Down Expand Up @@ -955,6 +957,17 @@ Planned support for responding to pull request comments with interactive command
@diffscope help # Show all commands
```

### Requested Reviewer Automation

Server webhook deployments can run reviews only when a specific reviewer is requested. For an org-level EvalOpsBot setup, subscribe the GitHub App or webhook to `pull_request` events and run the server with:

```bash
DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS=review_requested
DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS=EvalOpsBot
```

With that configuration, `pull_request.review_requested` starts a full DiffScope review only when GitHub reports `requested_reviewer.login` as `EvalOpsBot`.

### ✅ Feedback Loop (Reduce Repeated False Positives)

Use the feedback store to suppress comments you’ve already reviewed:
Expand Down
6 changes: 6 additions & 0 deletions charts/diffscope/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ metadata:
{{- include "diffscope.labels" . | nindent 4 }}
data:
DIFFSCOPE_MODEL: {{ .Values.diffscope.model | quote }}
{{- if .Values.diffscope.github.autoReviewEvents }}
DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS: {{ join "," .Values.diffscope.github.autoReviewEvents | quote }}
{{- end }}
{{- if .Values.diffscope.github.reviewRequestReviewers }}
DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS: {{ join "," .Values.diffscope.github.reviewRequestReviewers | quote }}
{{- end }}
{{- if .Values.diffscope.baseUrl }}
DIFFSCOPE_BASE_URL: {{ .Values.diffscope.baseUrl | quote }}
{{- else if .Values.ollama.enabled }}
Expand Down
8 changes: 8 additions & 0 deletions charts/diffscope/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ diffscope:
adapter: ""
# Base URL for LLM API (auto-set to Ollama service URL when ollama.enabled)
baseUrl: ""
github:
# pull_request actions that start server-side reviews.
# Set to ["review_requested"] with reviewRequestReviewers for on-demand org review bots.
autoReviewEvents:
- opened
- synchronize
# GitHub user logins whose requested reviews trigger review_requested automation.
reviewRequestReviewers: []
extraArgs: []

# -- API keys (stored in a Secret)
Expand Down
10 changes: 10 additions & 0 deletions docs/self-hosting.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ config:

extraEnv:
DIFFSCOPE_SERVER_API_KEY: ${DIFFSCOPE_SERVER_API_KEY}
DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS: review_requested
DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS: EvalOpsBot
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
```
Expand All @@ -89,14 +91,22 @@ For enterprise installs, prefer external secret injection over storing credentia
- `OPENAI_API_KEY`
- `OPENROUTER_API_KEY`
- `DIFFSCOPE_SERVER_API_KEY`
- `DIFFSCOPE_WEBHOOK_SECRET`
- `GITHUB_TOKEN`
- `DIFFSCOPE_GITHUB_APP_ID`
- `DIFFSCOPE_GITHUB_PRIVATE_KEY`
- `DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS`
- `DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS`
- `DIFFSCOPE_JIRA_BASE_URL`
- `DIFFSCOPE_JIRA_EMAIL`
- `DIFFSCOPE_JIRA_API_TOKEN`
- `DIFFSCOPE_LINEAR_API_KEY`

For an on-demand review bot, set `DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS=review_requested`
and `DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS=EvalOpsBot`. The GitHub webhook must
subscribe to `pull_request` events; DiffScope will ignore review requests for other
reviewers.

### Validation behavior

`diffscope doctor`, the server doctor endpoint, and startup warnings now surface configuration issues for:
Expand Down
190 changes: 188 additions & 2 deletions src/config.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use anyhow::Result;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use tracing::warn;

Expand Down Expand Up @@ -150,7 +150,7 @@ pub struct VaultConfig {
pub namespace: Option<String>,
}

#[derive(Debug, Clone, Serialize, Deserialize, Default)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GitHubConfig {
#[serde(default, rename = "github_token")]
pub token: Option<String>,
Expand All @@ -174,6 +174,37 @@ pub struct GitHubConfig {
/// Webhook secret for verifying GitHub webhook signatures.
#[serde(default, rename = "github_webhook_secret")]
pub webhook_secret: Option<String>,

/// pull_request actions that should start an automated review.
///
/// Supported values: opened, synchronize, reopened, review_requested.
#[serde(
default = "default_github_auto_review_events",
rename = "github_auto_review_events"
)]
pub auto_review_events: Vec<String>,

/// GitHub user logins that can trigger review_requested automation.
///
/// This is intentionally separate from auto_review_events so an org-level
/// webhook can listen to pull_request events without reviewing every PR.
#[serde(default, rename = "github_review_request_reviewers")]
pub review_request_reviewers: Vec<String>,
}

impl Default for GitHubConfig {
fn default() -> Self {
Self {
token: None,
app_id: None,
client_id: None,
client_secret: None,
private_key: None,
webhook_secret: None,
auto_review_events: default_github_auto_review_events(),
review_request_reviewers: Vec::new(),
}
}
}

#[derive(Debug, Clone, Serialize, Deserialize, Default)]
Expand Down Expand Up @@ -1034,6 +1065,16 @@ impl Config {
.ok()
.filter(|s| !s.trim().is_empty());
}
if let Ok(events) = std::env::var("DIFFSCOPE_GITHUB_AUTO_REVIEW_EVENTS") {
self.github.auto_review_events = split_csv_list(&events);
}
if self.github.review_request_reviewers.is_empty() {
self.github.review_request_reviewers =
std::env::var("DIFFSCOPE_GITHUB_REVIEW_REQUEST_REVIEWERS")
.ok()
.map(|reviewers| split_csv_list(&reviewers))
.unwrap_or_default();
}
if self.jira.base_url.is_none() {
self.jira.base_url = std::env::var("DIFFSCOPE_JIRA_BASE_URL")
.ok()
Expand Down Expand Up @@ -1090,6 +1131,8 @@ impl Config {
normalize_optional_trimmed(&mut self.github.client_secret);
normalize_optional_trimmed(&mut self.github.private_key);
normalize_optional_trimmed(&mut self.github.webhook_secret);
normalize_lowercase_list(&mut self.github.auto_review_events);
normalize_string_list(&mut self.github.review_request_reviewers);
normalize_optional_trimmed(&mut self.jira.base_url);
normalize_optional_trimmed(&mut self.jira.email);
normalize_optional_trimmed(&mut self.jira.api_token);
Expand Down Expand Up @@ -1701,6 +1744,32 @@ impl Config {
"github_client_secret is set without github_client_id. The device flow only uses github_client_id today.",
));
}
for event in &self.github.auto_review_events {
if !matches!(
event.as_str(),
"opened" | "synchronize" | "reopened" | "review_requested"
) {
issues.push(ConfigValidationIssue::warning(
"github_auto_review_events",
format!(
"Unsupported GitHub pull_request action '{}'; supported values are opened, synchronize, reopened, and review_requested.",
event
),
));
}
}
if self
.github
.auto_review_events
.iter()
.any(|event| event == "review_requested")
&& self.github.review_request_reviewers.is_empty()
{
issues.push(ConfigValidationIssue::warning(
"github_review_request_reviewers",
"review_requested automation is enabled but no requested reviewer logins are configured.",
));
}

let jira_fields = [
self.jira.base_url.as_ref(),
Expand Down Expand Up @@ -1827,6 +1896,37 @@ fn normalize_optional_trimmed(value: &mut Option<String>) {
.map(ToOwned::to_owned);
}

fn split_csv_list(value: &str) -> Vec<String> {
value.split(',').map(ToOwned::to_owned).collect()
}

fn normalize_string_list(values: &mut Vec<String>) {
let mut seen = HashSet::new();
values.retain_mut(|value| {
let trimmed = value.trim();
if trimmed.is_empty() {
return false;
}

let normalized_key = trimmed.to_ascii_lowercase();
if !seen.insert(normalized_key) {
return false;
}

if trimmed.len() != value.len() {
*value = trimmed.to_string();
}
true
});
}

fn normalize_lowercase_list(values: &mut Vec<String>) {
normalize_string_list(values);
for value in values {
*value = value.to_ascii_lowercase();
}
}

fn apply_provider_env_api_key_fallback(
providers: &mut HashMap<String, ProviderConfig>,
provider_name: &str,
Expand Down Expand Up @@ -1903,6 +2003,10 @@ fn default_temperature() -> f32 {
0.2
}

fn default_github_auto_review_events() -> Vec<String> {
vec!["opened".to_string(), "synchronize".to_string()]
}

fn default_max_tokens() -> usize {
4000
}
Expand Down Expand Up @@ -2927,6 +3031,88 @@ auditing_model_role: primary
assert!(issues.iter().any(|issue| issue.field == "vault"));
}

#[test]
fn test_github_auto_review_events_default_to_opened_and_synchronize() {
let config = Config::default();

assert_eq!(
config.github.auto_review_events,
vec!["opened".to_string(), "synchronize".to_string()]
);
}

#[test]
fn test_config_deserialize_github_review_request_controls() {
let config: Config = serde_yaml::from_str(
r#"
github_auto_review_events:
- review_requested
github_review_request_reviewers:
- EvalOpsBot
"#,
)
.unwrap();

assert_eq!(
config.github.auto_review_events,
vec!["review_requested".to_string()]
);
assert_eq!(
config.github.review_request_reviewers,
vec!["EvalOpsBot".to_string()]
);
}

#[test]
fn test_normalize_github_review_request_controls() {
let mut config = Config {
github: GitHubConfig {
auto_review_events: vec![
" Review_Requested ".to_string(),
"review_requested".to_string(),
"opened".to_string(),
"".to_string(),
],
review_request_reviewers: vec![
" EvalOpsBot ".to_string(),
"evalopsbot".to_string(),
"AnotherBot".to_string(),
"".to_string(),
],
..GitHubConfig::default()
},
..Config::default()
};

config.normalize();

assert_eq!(
config.github.auto_review_events,
vec!["review_requested".to_string(), "opened".to_string()]
);
assert_eq!(
config.github.review_request_reviewers,
vec!["EvalOpsBot".to_string(), "AnotherBot".to_string()]
);
}

#[test]
fn test_validation_warns_when_review_request_enabled_without_reviewers() {
let config = Config {
github: GitHubConfig {
auto_review_events: vec!["review_requested".to_string()],
review_request_reviewers: Vec::new(),
..GitHubConfig::default()
},
..Config::default()
};

let issues = config.validation_issues();
assert!(issues
.iter()
.any(|issue| issue.field == "github_review_request_reviewers"));
}

#[test]
fn test_normalize_rejects_embedding_for_generation_verification_and_auditing() {
let mut config = Config {
Expand Down
Loading
Loading