Contributing to LLM Research: A Beginner's Guide

I shipped a small CLI agent that turns any LLM paper abstract into a concrete contribution plan with starter code. If you are new to research and tired of reading papers without knowing what to build next, this tool closes the loop between reading and doing. We will build it in four steps using Oxlo.ai and the OpenAI SDK.

What you'll need

• Python 3.10 or newer.
• The OpenAI SDK: pip install openai.
• An Oxlo.ai API key from https://portal.oxlo.ai. I use Oxlo.ai because its request-based pricing means a 4,000-token abstract costs the same as a 200-token prompt. That matters when you are iterating on long inputs.
• A paper abstract saved as a plain text file named paper.txt.

Step 1: Set up the Oxlo.ai client

I keep credentials out of source, so this snippet pulls the key from the environment and points the OpenAI SDK at Oxlo.ai. No other config is needed. The SDK is a drop-in replacement, which means the same code works for prototyping and production without vendor lock-in.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

Step 2: Define the research agent prompt

The system prompt is the only real logic in this project. I iterated on this extensively. My first version simply asked for research ideas, and the model returned vague essays. I fixed that by forcing five specific markdown sections and explicitly forbidding fluff. The constraint to output a code skeleton is the most important part. It gives the beginner a compile-checked artifact to start from instead of a blank page.

The "Gap" section is the hardest to get right. I added the instruction "Avoid vague statements" because otherwise the model outputs things like "future work could explore efficiency," which is useless. By demanding specificity, the model instead says something like "the attention matrix is dense and scales quadratically," which is a real mechanical constraint you can attack.

I run this on kimi-k2.6 through Oxlo.ai. Its 131K context window easily fits long abstracts plus the system prompt, and its reasoning quality is strong enough to generate valid PyTorch skeletons from just an abstract. If you are on the Oxlo.ai free tier, deepseek-v3.2 is a solid alternative for this workload.

SYSTEM_PROMPT = """You are a senior ML researcher helping a junior contributor find a concrete entry point into LLM research.

When given a paper title and abstract, produce a structured analysis with exactly these sections:

1. Summary: One paragraph explaining the core contribution.
2. Gap: Identify one specific limitation that a newcomer could plausibly address. Avoid vague statements. Be specific about what is missing.
3. Experiment: Propose a small, tractable experiment to investigate the gap. Include a hypothesis, dataset suggestion, and evaluation metric.
4. Code skeleton: Provide a minimal Python script that sets up the experiment. Use PyTorch or Hugging Face Transformers. Keep it under 40 lines.
5. Next read: Suggest two related papers to read.

Write in markdown. Be concise and actionable."""

Step 3: Build the analysis pipeline

This function wraps the API call. We format the title and abstract into a single user message. I set temperature to 0.3 because I want a reproducible experiment plan, not creative speculation. The call itself is identical to the standard OpenAI pattern, just routed through Oxlo.ai.

Notice that I do not parse JSON. I ask for markdown because it is human-readable and easy to edit. If the model hallucinates a library function, I can fix it in the markdown file without fighting a JSON schema. This is a deliberate trade-off. Structured output is great for machines, but for research prototyping, I want human inspectability first.

def analyze_paper(title: str, abstract: str) -> str:
    user_message = f"Title: {title}\n\nAbstract: {abstract}"

    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.3,
    )

    return response.choices[0].message.content

Step 4: Parse and save results

I do not want to re-run the script just to see the output again, so this step reads the local paper.txt and writes the result to contribution_plan.md. I chose a flat text file over JSON or YAML because researchers already have abstracts in plain text. The script splits on the first newline. If your title contains a newline, replace it with a space first. The early exit keeps the CLI clean if the file is missing.

The split logic is intentionally minimal. I do not use a CSV parser or a front-matter parser because those add dependencies. A single newline split is robust enough for 99% of arXiv copy-paste jobs. If you need to process PDFs later, you can swap this line for a PyPDF2 call without touching the Oxlo.ai integration. I also use pathlib because it handles encoding and cross-platform paths without extra imports. Writing to disk means you can git commit both the input abstract and the generated plan, creating a reproducible research log from day one.

import sys
from pathlib import Path

def main():
    paper_path = Path("paper.txt")

    if not paper_path.exists():
        print("Create paper.txt with the title on line 1 and the abstract on the remaining lines.")
        sys.exit(1)

    raw = paper_path.read_text(encoding="utf-8").strip()
    lines = raw.split("\n", 1)
    title = lines[0].strip()
    abstract = lines[1].strip() if len(lines) > 1 else ""

    print(f"Analyzing: {title}")
    analysis = analyze_paper(title, abstract)

    out_path = Path("contribution_plan.md")
    out_path.write_text(analysis, encoding="utf-8")
    print(f"Saved contribution plan to {out_path}")

if __name__ == "__main__":
    main()

Run it

Create a file named paper.txt in the same directory. Here is an example using a well-known transformer paper.

Attention Is All You Need
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Run the script from your terminal.

$ python research_agent.py
Analyzing: Attention Is All You Need
Saved contribution plan to contribution_plan.md

The agent returns a full markdown plan. Here is an excerpt from a real run against Oxlo.ai. The Gap is narrow enough for a single contributor, and the Experiment is designed to fit on a single GPU over a weekend.

## Summary
The authors introduce the Transformer, an encoder-decoder architecture that replaces recurrence and convolution with multi-head self-attention. This enables massive parallelization and yields strong results on machine translation and parsing.

## Gap
The paper evaluates translation and parsing, but it does not measure how the pure attention mechanism scales to long-context language modeling beyond a few hundred tokens. The quadratic cost of full attention is noted only in passing, not benchmarked on long documents.

## Experiment
Hypothesis: Replacing full self-attention with a sparse or sliding-window attention pattern will retain 95% of perplexity on the PG-19 long-form books dataset while reducing memory usage by 50%.
Dataset: PG-19 test split.
Metric: Perplexity and peak GPU memory.
Baseline: A 6-layer Transformer with standard attention trained for 10k steps.

## Code skeleton
```python
import torch
from transformers import GPT2Config, GPT2LMHeadModel, Trainer, TrainingArguments

config = GPT2Config(n_layer=6, n_head=8, n_embd=512)
model = GPT2LMHeadModel(config)

# TODO: implement sliding-window attention mask here
# and patch model.forward to use it

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=1,
    per_device_train_batch_size=4,
    logging_steps=100,
)

trainer = Trainer(model=model, args=training_args, train_dataset=...)
trainer.train()
```

## Next read
1. "Generating Wikipedia by Summarizing Long Sequences" (PG-19 introduction).
2. "Longformer: The Long-Document Transformer" (efficient attention patterns).

When you read the output, check that the Experiment section mentions a concrete dataset and metric. If it suggests something too large, like "train a 70B model," edit the system prompt to add a constraint like "propose an experiment that fits on a single A100 40GB GPU." The model will respect that budget.

Next steps

Push the generated code skeleton into a GitHub repo and use it as the seed for your first reproducibility attempt. Then share the result on Hugging Face or arXiv as a technical report. If you want to scale up, loop over a directory of abstracts and use Oxlo.ai's request-based pricing to analyze dozens of papers. Because Oxlo.ai charges per request, a long abstract costs the same as a short one, so batch processing a reading list is predictable and cheap.

You can also pipe the Gap and Experiment sections to qwen-3-32b on Oxlo.ai to generate a deeper ablation plan and statistical power analysis. The OpenAI-compatible client makes swapping models trivial, so you can chain agents without rewriting any networking code. Another direction is to turn this into a web app with Streamlit. You would replace the file read with a text area and call the same analyze_paper function on every submit. Because Oxlo.ai has no cold starts on popular models, the user gets an answer immediately instead of