Skip to content

Commit

Permalink
feat: docs for pr review bot (#613)
Browse files Browse the repository at this point in the history
# Motivation

<!-- Why is this change necessary? -->

# Content

<!-- Please include a summary of the change -->

# Testing

<!-- How was the change tested? -->

# Please check the following before marking your PR as ready for review

- [x ] I have added tests for my changes
- [ x] I have updated the documentation or added new documentation as
needed

---------

Co-authored-by: kopekC <[email protected]>
  • Loading branch information
kopekC and kopekC authored Feb 21, 2025
1 parent b2f5d1a commit 17fa0bc
Show file tree
Hide file tree
Showing 9 changed files with 3,160 additions and 198 deletions.
11 changes: 11 additions & 0 deletions codegen-examples/examples/pr_review_bot/.env.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
GITHUB_TOKEN="..."
# ====[ MODEL PROVIDERS ]===

ANTHROPIC_API_KEY="..."
OPENAI_API_KEY="..."
GOOGLE_API_KEY="..."
GEMINI_API_KEY="..."

# ====[ SLACK ]===
SLACK_SIGNING_SECRET="..."
SLACK_BOT_TOKEN="..."
140 changes: 27 additions & 113 deletions codegen-examples/examples/pr_review_bot/README.md
Original file line number Diff line number Diff line change
@@ -1,129 +1,43 @@
# AI-Powered Pull Request Review Bot

This example demonstrates how to use Codegen to create an intelligent PR review bot that analyzes code changes and their dependencies to provide comprehensive code reviews. The bot uses GPT-4 to generate contextual feedback based on modified code and its relationships.
This example project demonstrates how to deploy an agentic bot that automatically reviews GitHub Pull Requests. The bot analyzes code changes and their dependencies to provide comprehensive code reviews using AI, considering both direct modifications and their impact on the codebase.

> [!NOTE]
> This codemod helps development teams by providing automated, context-aware code reviews that consider both direct and indirect code dependencies.
## Prerequisites

## How the PR Review Bot Works
Before running this application, you'll need the following API tokens and credentials:

The script analyzes pull requests in several key steps:
- GitHub API Token
- Anthropic API Token
- GitHub Repository Access

1. **Symbol Analysis**
## Setup

```python
modified_symbols = codebase.get_modified_symbols_in_pr(pr_number)
for symbol in modified_symbols:
deps = codebase.get_symbol_dependencies(symbol, max_depth=2)
rev_deps = codebase.get_symbol_dependents(symbol, max_depth=2)
```

- Identifies modified symbols in the PR
- Analyzes dependencies up to 2 levels deep
- Tracks reverse dependencies (symbols that depend on changes)

1. **Context Building**

```python
context = {"pr_title": pr.title, "pr_body": pr.body, "modified_symbols": [...], "context_symbols": [...]}
```

- Gathers PR metadata
- Collects modified code content
- Includes relevant dependency context

1. **AI Review Generation**

```python
review = codebase.ai_client.llm_query_with_retry(messages=[...], model="gpt-4", max_tokens=2000)
```

- Uses GPT-4 for analysis
- Generates comprehensive review feedback
- Considers full context of changes

## Why This Makes Code Review Better

1. **Context-Aware Analysis**

- Understands code dependencies
- Considers impact of changes
- Reviews code in proper context

1. **Comprehensive Review**

- Analyzes direct modifications
- Evaluates dependency impact
- Suggests improvements

1. **Consistent Feedback**

- Structured review format
- Thorough analysis every time
- Scalable review process

## Review Output Format

The bot provides structured feedback including:
1. Clone the repository
1. Set up your environment variables in a `.env` file:

```env
GITHUB_TOKEN=your_github_token
ANTHROPIC_API_KEY=your_anthropic_token
GITHUB_REPO_OWNER=your_repo_owner
GITHUB_REPO_NAME=your_repo_name
GITHUB_PR_NUMBER=your_pr_number
```
1. Overall Assessment
- High-level review of changes
- Impact analysis
2. Specific Code Feedback
- Detailed code comments
- Style suggestions
- Best practices
3. Potential Issues
- Security concerns
- Performance impacts
- Edge cases
4. Dependency Analysis
- Impact on dependent code
- Breaking changes
- Integration considerations
```

## Key Benefits to Note

1. **Better Code Quality**

- Thorough code analysis
- Consistent review standards
- Early issue detection

1. **Time Savings**

- Automated initial review
- Quick feedback loop
- Reduced review burden

1. **Knowledge Sharing**

- Educational feedback
- Best practice suggestions
- Team learning

## Configuration Options

You can customize the review by:
## Features

- Adjusting dependency depth
- Modifying the AI prompt
- Changing the review focus areas
- Tuning the GPT-4 parameters
- Automated PR code review using AI
- Deep dependency analysis of code changes
- Context-aware feedback generation
- Structured review format with actionable insights
- Integration with GitHub PR system

## Learn More
## Usage

- [Codegen Documentation](https://docs.codegen.com)
- [OpenAI API Documentation](https://platform.openai.com/docs/api-reference)
- [GitHub API Documentation](https://docs.github.com/en/rest)
- [Codegen llm integration](https://docs.codegen.com/building-with-codegen/calling-out-to-llms)
1. `uv sync`
1. `uv run modal deploy app.py`
- This will deploy a modal app that can be triggered to review PRs
1. Create or update a PR to trigger the review bot

## Contributing

Feel free to submit issues and enhancement requests! Contributions to improve the review bot's capabilities are welcome.
Contributions are welcome! Please feel free to submit a Pull Request.
66 changes: 66 additions & 0 deletions codegen-examples/examples/pr_review_bot/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import logging
from logging import getLogger
import modal
from codegen.extensions.events.app import CodegenApp
from fastapi import Request
from codegen.extensions.github.types.events.pull_request import PullRequestLabeledEvent, PullRequestUnlabeledEvent
from helpers import remove_bot_comments, pr_review_agent

logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s")
logger = getLogger(__name__)

REPO_URL = "https://github.com/codegen-sh/codegen-sdk.git"
COMMIT_ID = "20ba52b263ba8bab552b5fb6f68ca3667c0309fb"

base_image = (
modal.Image.debian_slim(python_version="3.12")
.apt_install("git")
.pip_install(
# =====[ Codegen ]=====
# "codegen>=0.18",
f"git+{REPO_URL}@{COMMIT_ID}",
# =====[ Rest ]=====
"openai>=1.1.0",
"fastapi[standard]",
"slack_sdk",
)
)

app = CodegenApp(name="github", image=base_image, modal_api_key="")


@app.github.event("pull_request:labeled")
def handle_labeled(event: PullRequestLabeledEvent):
logger.info("[PULL_REQUEST:LABELED] Received pull request labeled event")
logger.info(f"PR #{event.number} labeled with: {event.label.name}")
logger.info(f"PR title: {event.pull_request.title}")
# app.slack.client.chat_postMessage(
# channel="C08DPPSL1CG",
# text=f"PR #{event.number} labeled with: {event.label.name}",
# )
if event.label.name == "Codegen":
app.slack.client.chat_postMessage(
channel="C08DPPSL1CG",
text=f"PR #{event.number} labeled with: {event.label.name}, waking up CodegenBot and starting review",
)

logger.info(f"PR ID: {event.pull_request.id}")
logger.info(f"PR title: {event.pull_request.title}")
logger.info(f"pr number: {event.number}, ")
pr_review_agent(event)


@app.github.event("pull_request:unlabeled")
def handle_unlabeled(event: PullRequestUnlabeledEvent):
logger.info("unlabeled")
logger.info(event.action)
logger.info(event.label.name)
if event.label.name == "Codegen":
remove_bot_comments(event)


@app.function(secrets=[modal.Secret.from_dotenv()])
@modal.web_endpoint(method="POST")
def entrypoint(event: dict, request: Request):
logger.info("[OUTER] Received GitHub webhook")
return app.github.handle(event, request)
90 changes: 90 additions & 0 deletions codegen-examples/examples/pr_review_bot/helpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
from github import Github
from codegen.extensions.github.types.events.pull_request import PullRequestUnlabeledEvent
from logging import getLogger

import os

from codegen import Codebase

from codegen.extensions.github.types.events.pull_request import PullRequestLabeledEvent
from codegen.configs.models.secrets import SecretsConfig
from codegen import CodeAgent

from codegen.extensions.langchain.tools import (
# Github
GithubViewPRTool,
GithubCreatePRCommentTool,
GithubCreatePRReviewCommentTool,
)

from dotenv import load_dotenv
import logging

load_dotenv()

logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s")
logger = getLogger(__name__)


def remove_bot_comments(event: PullRequestUnlabeledEvent):
g = Github(os.getenv("GITHUB_API_KEY"))
logger.info(f"{event.organization.login}/{event.repository.name}")
repo = g.get_repo(f"{event.organization.login}/{event.repository.name}")

pr = repo.get_pull(int(event.number))
comments = pr.get_comments()
if comments:
for comment in comments:
logger.info("removing comment")
logger.info(comment.user.login)
if comment.user.login == "codegen-team":
comment.delete()
reviews = pr.get_reviews()

if reviews:
for review in reviews:
logger.info("removing review")
logger.info(review.user.login)
if review.user.login == "codegen-team":
review.delete()

issue_comments = pr.get_issue_comments()
if issue_comments:
for comment in issue_comments:
logger.info("removing comment")
logger.info(comment.user.login)
if comment.user.login == "codegen-team":
comment.delete()


def pr_review_agent(event: PullRequestLabeledEvent) -> None:
# Pull a subset of SWE bench
repo_str = f"{event.organization.login}/{event.repository.name}"
codebase = Codebase.from_repo(repo_str, language="python", secrets=SecretsConfig(github_token=os.environ["GITHUB_TOKEN"]))
review_atention_message = "CodegenBot is starting to review the PR please wait..."
comment = codebase._op.create_pr_comment(event.number, review_atention_message)
# Define tools first
pr_tools = [
GithubViewPRTool(codebase),
GithubCreatePRCommentTool(codebase),
GithubCreatePRReviewCommentTool(codebase),
]

# Create agent with the defined tools
agent = CodeAgent(codebase=codebase, tools=pr_tools)

# Using a prompt from SWE Bench
prompt = f"""
Hey CodegenBot!
Here's a SWE task for you. Please Review this pull request!
{event.pull_request.url}
Do not terminate until have reviewed the pull request and are satisfied with your review.
Review this Pull request like the señor ingenier you are
be explicit about the changes, produce a short summary, and point out possible improvements where pressent dont be self congratulatory stick to the facts
use the tools at your disposal to create propper pr reviews include code snippets if needed, and suggest improvements if feel its necesary
"""
# Run the agent
agent.run(prompt)
comment.delete()
12 changes: 12 additions & 0 deletions codegen-examples/examples/pr_review_bot/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[project]
name = "ticket-to-pr"
version = "0.1.0"
description = "A example implementation of a agentic linear bot that can create PRs from linear tickets"
readme = "README.md"
requires-python = ">=3.12, <3.14"
dependencies = [
"codegen==0.31.1",
"fastapi>=0.115.8",
"modal>=0.73.51",
"pydantic>=2.10.6",
]
Loading

0 comments on commit 17fa0bc

Please sign in to comment.