以 LLM 打造助手，輕鬆撰寫 PR 描述

You know that feeling. You've just poured your soul into a feature, pushed your branch, and then... the dreaded blank PR description box. Your brain is fried. You just want to ship it, but you should write a good description. Explaining your changes, the 'why,' the 'what.' It's a mental tax I've always hated.

After one particularly grueling merge week, I decided enough was enough. What if an LLM could at least draft these things for me? I wanted something that wasn't just a generic recap but actually understood the context of my changes. So, over a recent weekend, I set out to build it. Here’s how it went down:

1. The "Duh" Moment & First Try

My initial thought was, 'Hey, LLMs are good at summarizing code, right?' What if I just fed it the raw git diff? Simple. Elegant. Probably naive. But that's usually where my best (and worst) ideas start. I figured the minimal effort approach was the best way to get started and see what broke.

2. Grabbing the Diff

Getting the diff programmatically is pretty straightforward. A quick subprocess.run call in Python did the trick. I usually compare against main for PRs, so it was git diff main...HEAD. The ... is crucial here to get all the commits on the current branch, not just the changes since the branch diverged from main.

python
import subprocess

def get_git_diff():
try:
# Compare current branch HEAD against main
result = subprocess.run(
['git', 'diff', 'main...HEAD'],
capture_output=True, text=True, check=True
)
return result.stdout
except subprocess.CalledProcessError as e:
print(f"Error getting diff: {e.stderr}")
return None

I was using Python 3.10 for this little script, just because that's what my pyenv default was set to.

3. Initial Prompting & The LLM

For the LLM, I picked gpt-3.5-turbo-0125 for its speed and cost-effectiveness for initial experiments. My first prompt was embarrassingly simple: Summarize these git changes into a pull request description, explaining what was changed and why. Keep it concise. Then I just appended the diff directly after that.

4. The Unexpected Hiccup (My First Disappointment)

This is where I ran into my first major roadblock. I was expecting magic, but I got... vanilla. The LLM would spit out descriptions that were technically correct, but completely lacked context. It would say, 'Added a new function to handle user authentication,' but it wouldn't know why that function was needed, or what problem it solved from a higher level.

It was too literal to the code, like a junior dev reading lines and rephrasing them. For example, if I had a feature branch that fixed a bug where users couldn't log in after a password reset, the LLM would say: 'Modified auth_service.py to update login_user function parameters.' No mention of 'bug fix' or 'password reset flow.' It was just a dry recap of the code changes. I'd spent about 2 hours on this initial setup, only to get something barely better than me just looking at the diff.

5. Injecting Context – The "Why"

I quickly realized the LLM only knows what you feed it. The git diff shows what changed, but not why. The 'why' often lives in commit messages, or maybe even in a Jira ticket ID if you're disciplined. I started experimenting with adding the commit messages from the current branch.

python
def get_git_log():
try:
# Get commit messages for current branch (since diverging from main)
result = subprocess.run(
['git', 'log', 'main...HEAD', '--pretty=format:%s%n%b'],
capture_output=True, text=True, check=True
)
return result.stdout
except subprocess.CalledProcessError as e:
print(f"Error getting log: {e.stderr}")
return None

Then I updated my prompt. Instead of just the diff, I added a section for Commit Messages and Git Diff and tweaked the prompt instruction: Analyze the following commit messages and git diff. Based on this information, generate a detailed and context-aware pull request description. Focus on the problem being solved, the solution implemented, and any relevant impact. Include a suggested title and a summary section. This made a huge difference. Suddenly, the LLM started connecting the dots between the commits and the code changes.

6. The Working Script and Beyond

Now I had something genuinely useful. I wrapped it all up in a simple Python script that takes a few arguments (like --clipboard to copy directly, or --print to just show). I added some markdown formatting to the LLM's output request, asking for sections like 'Title', 'Summary', 'Problem', 'Solution', and 'Changes'.

python

Simplified example of combining diff and log for LLM input

full_context = f"Commit Messages:\n{get_git_log()}\n\nGit Diff:\n{get_git_diff()}"
prompt = f"""
Analyze the following commit messages and git diff. Based on this information, generate a detailed and context-aware pull request description.
Focus on the problem being solved, the solution implemented, and any relevant impact.
Include a suggested title and the following sections:

• Summary: A high-level overview of the changes.
• Problem: What issue does this PR address?
• Solution: How does this PR solve the problem?
• Changes: A brief bulleted list of technical changes.

Here's the context:\n\n{full_context}
"""

Then send 'prompt' to LLM API and process the response

This little automation isn't going to write your entire codebase, but it's a huge boost to developer experience. It gets me 80% of the way there, and I just fill in the last 20% with specific details or design decisions. It’s wild how much difference adding those commit messages made. Next up, maybe integrating it directly into my editor or as a pre-commit hook that generates a draft PR description right before I push. No more blank boxes for me!