DataLinter 檢查並改善數據科學工作流程

by Corneliu Cofaru, Adrian Iurov at 0x0α Research.

In this article we explore¹ how LLM-generated code can be reliably
used in data science experiments and workflows. The aim is to
illustrate the potential of combining language models with code and
data quality tools. We devise a simple regression experiment in which
a LLM generates code in R, starting from anonymized tabular data based
on the ’Ames housing’ dataset. The generated code is then executed and
interactively analyzed with a linter capable of running checks on both
data results and code simultaneously. Results from the linter are then
fed back into the LLM and new code is generated which in its turn
produces an improved model with a lower error than the original one.

The theory

LLMs are starting to be used in generating analysis code in data science
workflows with seemingly good results [1, 2]. The general approach to obtain
the code relies on building first a prompt that describes in more or
less objective terms the dataset, its properties, column names and
meanings, etc. Such an LLM-based data analysis workflow is depicted in
the image below:

The workflow can be summarized as follows:

construct a code-generation LLM prompt from dataset properties
run the LLM using the prompt to obtain R code
execute the R code and obtain original RMSE

The results of this approach seem to be encouraging and indicate that
there is definitely a pathway forward for the use of LLMs in data
science. However, LLMs have the limitation that they cannot perform
reliably simple numerical operations - counting, averaging, that are
crucial in data analysis and trivial for general purpose programs to
perform. This becomes a problem in code generation where the code
correctness is dependent on exact data properties that are extracted
using such operations. Moreover, the code may contain operations that
modify the data in ways that make it irrelevant with respect to the
initial prompt used to generate the code itself. Lastly, data sizes and
complexity matter - one cannot simply feed whole datasets to the LLM and
expect reliable results. Even though this could be possible due to some
learnt structures that associate data properties to modelling code
within the language model it is definitely the exception and not the
rule. These considerations lead to a more nuanced approach in using
LLMs, one in which code analysis, data analysis and corrective feedback
on code and/or data using specialized programs becomes a necessary
requirement.

In data science, the experimental design challenge is complicated by the
fact that more often than not, model correctness and performance depend
on both data and code. This is an additional argument for the necessity
of a feedback loop with information related to data and the
LLM-generated code that models it. Here we use DataLinter [3], a tool we
developed that can find issues by analyzing simultaneously data and
code. If we add the linter-based LLM feedback loop to the original data
analysis workflow, we obtain a new one, represented in the figure below:

The new workflow, that integrates data and code corrections, becomes:

construct a code-generation LLM prompt from dataset properties
(same as before)
run the LLM using the prompt to obtain R code (same as before)
execute the R code and obtain original RMSE (same as before)
run DataLinter on both data and modelling code
construct a code-improvement LLM prompt using linter output
run LLM using the new prompt and obtain new R code
execute the R code and obtain new RMSE
compare the two RMSEs

In practice

To implement the LLM data analysis flow and feedback loops, we start by
writing a simple Python script that extracts simple data statistics:
column names, element types, number of NAs/Nulls for each column as
well as maximum, minimum, median and mean values (for numerical columns)
and minimum and maximum element length (for string columns). The dataset
column statistics are interpolated into a fixed-format string yielding
the initial code-generation LLM prompt. An example can be viewed in this
notebook.
The prompt can be pasted into any LLM. We use Claude Opus 4.6 as R code
generator². After running the code in VSCode, we obtain a RMSE of
27895.36.

To lint the modelling code and data, we use the DataLinter and its
VSCode plugin for R. The linter is designed to work with data, code or
both. Under the hood, the VSCode plugin sends to a linting server a
snippet of code and any data variable identified. This is done simply by
selecting code with the mouse and a right-click to the DataLinter menu
option. Data extraction works by parsing the selected code and looking
for data = ... patterns present inside function calls. In this case,
the contents of variable train are sent along with the code
model <- lm(x81 ., data = train).

The result is six problematic issues, shown in the figure above. The linter output is then used to
construct the code-improvement LLM prompt and a new version of the R
code³. The newly obtained code is ran and, after addressing issues
such as numeric scale imbalance and target skew via standardization and
a log transform, the RMSE improved!

Closing remarks

To wrap things up, we are happy to have confirmed our initial assumption
that code and data quality tools fit well and can improve LLM-based data
science workflows. In this case, with very little effort we were able to
improve model performance in our regression experiment. The only
necessary condition to ensure correctness and performance of the code is
for the linter to have a good coverage over the space of potential
problems that may appear. This is in itself a big challenge and
interesting area of research, especially for complex problematic
patterns such as data leakage. We hope to address and tackle
successfully these issues in future articles.

References

Reuben Sarwal et al. Benchmarking large language models for predictive modeling in biomedical research with a focus on reproductive health. Cell Reports Medicine, vol. 7, no. 2, pages 102594, 2026.
Robert Vacareanu, Vlad Andrei Negru, Vasile Suciu and Mihai Surdeanu. From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples. First Conference on Language Modeling, 2024.
Corneliu Cofaru. DataLinter on GitHub, 2026.
Ivan Zalesskiy. Best Local LLM for Coding in 2026: Developer's Guide.