Browse State-of-the-Art › Data Valuation
Data Valuation
53 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Data valuation in machine learning tries to determine the worth of data, or data sets, for downstream tasks. Some methods are task-agnostic and consider datasets as a whole, mostly for decision making in data markets. These look at distributional distances between samples. More often, methods look at how individual points affect performance of specific machine learning models. They assign a scalar to each element of a training set which reflects its contribution to the final performance of some model trained on it. Some concepts of value depend on a specific model of interest, others are model-agnostic.
Concepts of the usefulness of a datum or its influence on the outcome of a prediction have a long history in statistics and ML, in particular through the notion of the influence function. However, it has only been recently that rigorous and practical notions of value for data, and in particular data-sets, have appeared in the ML literature, often based on concepts from collaborative game theory, but also from generalization estimates of neural networks, or optimal transport theory, among others.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 119 · with a code link: 53 · where Syntology ran a sample: 21 (17 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (21 of 119 tagged: 17 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 119, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
5 Apr 2019 6 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)As data becomes the fuel driving technological and economic growth, a fundamental challenge is how to quantify the value of data in algorithmic predictions and decisions.
-
29 Jan 2024 3 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 1 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)Many tasks in explainable machine learning, such as data valuation and feature attribution, perform expensive computation for each data point and are intractable for large datasets.
-
11 Feb 2022 3 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)Over the last few years, the Shapley value, a solution concept from cooperative game theory, has found numerous applications in machine learning.
-
22 Aug 2019 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 2 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)The most surprising result is that for unweighted KNN classifiers and regressors, the Shapley value of all N data points can be computed, exactly, in O(NlogN) time -- an exponential improvement on computational…
-
7 Aug 2024 2 repositories listedData valuation has emerged as a powerful framework for quantifying each datum's contribution to the training of a machine learning model.
-
17 Jun 2024 2 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples)Data Shapley, a landmark study on data valuation, advances this understanding by assessing the contribution of each datum to model performance.
-
18 Jun 2023 2 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)Assessing the quality and impact of individual data points is critical for improving model performance and mitigating undesirable biases within the training dataset.
-
16 Apr 2023 2 repositories listedAs a result, it has been recognized as infeasible to apply to large datasets.
-
13 Nov 2022 2 repositories listedOur theoretical analysis shows the proposed value function is (essentially) the unique function that satisfies two desirable properties for evaluating data values in classification.
-
30 May 2022 2 repositories listedTo address this challenge, we introduce the concept of safety margin, which measures the robustness of a data value notion.
-
26 Oct 2021 2 repositories listedData Shapley has recently been proposed as a principled framework to quantify the contribution of individual datum in machine learning.
-
25 Sep 2019 2 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)To adaptively learn data values jointly with the target task predictor model, we propose a meta learning framework which we name Data Valuation using Reinforcement Learning (DVRL).
-
25 May 2025 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Data Shapley is an important tool for data valuation, which quantifies the contribution of individual data points to machine learning models.
-
23 Mar 2025 1 repository listedGraph Neural Networks (GNNs) have demonstrated remarkable performance in various graph-based machine learning tasks, yet evaluating the importance of neighbors of testing nodes remains largely unexplored due to the…
-
9 Mar 2025 1 repository listedHowever, in critical ML applications such as data valuation and feature attribution, the uniform weighting of Shapley values across subset cardinalities leads to unintuitive credit assignments.
-
2 Mar 2025 1 repository listedLarge Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited budget.
-
22 Feb 2025 1 repository listedData valuation is increasingly used in machine learning (ML) to decide the fair compensation for data owners and identify valuable or harmful data for improving ML models.
-
14 Feb 2025 1 repository listedParticularly, recent works have proposed various metrics and algorithms to calculate the influence of data using language models, which do not scale well with large models and datasets.
-
12 Feb 2025 1 repository listedUser behavior records serve as the foundation for recommender systems.
-
3 Feb 2025 1 repository listedFine-tuning large language models (LLMs) is often constrained by the computational costs of processing massive datasets.
-
5 Dec 2024 1 repository listedAssessing the importance of individual training samples is a key challenge in machine learning.
-
1 Nov 2024 1 repository listedThe Shapley value, a well-established concept from cooperative game theory, has been widely adopted to assess the contribution of individual data sources in supervised machine learning.
-
31 Oct 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)Particularly, our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values, including the well-known Shapley value, both theoretically and empirically.
-
6 Oct 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (of 13 harvested samples)Data valuation is a class of techniques for quantitatively assessing the value of data for applications like pricing in data marketplaces.
-
2 Oct 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)In this work, we introduce shapiq, an open-source Python package that unifies state-of-the-art algorithms to efficiently compute SVs and any-order SIs in an application-agnostic framework.
-
1 Oct 2024 1 repository listedSynthetic data generation has been proven successful in improving model performance and robustness in the context of scarce or low-quality data.
-
8 Sep 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)Influence Functions are a standard tool for attributing predictions to training data in a principled manner and are widely used in applications such as data valuation and fairness.
-
17 Jul 2024 1 repository listedData valuation quantifies the value of training data, and is used for data attribution (i. e., determining the contribution of training data towards model predictions), and data selection; both of which are important…
-
1 Jun 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL.
-
30 May 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)We introduce a new perspective by investigating scaling behavior for the value of individual data points: we find that a data point's contribution to model's performance shrinks predictably with the size of the dataset…
-
22 May 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
13 May 2024 1 repository listed
-
16 Mar 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 7 where Syntology's instrument failed) · 0 unverified (of 12 harvested samples) · 1 pointer-only (licence)
-
2 Feb 2024 1 repository listed
-
18 Dec 2023 1 repository listed
-
9 Nov 2023 1 repository listed
-
9 Nov 2023 1 repository listed
-
25 Jun 2023 1 repository listed
-
18 Jun 2023 1 repository listed
-
16 Jun 2023 1 repository listed
-
2 May 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
1 May 2023 1 repository listed
-
28 Apr 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
9 Apr 2023 1 repository listed
-
3 Mar 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
3 Jan 2023 1 repository listed
-
1 Dec 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
30 Oct 2022 1 repository listed
-
17 Dec 2021 1 repository listed Syntology official (archive's flag): 16 ran · 16 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 0 honoured, 0 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 18 harvested samples)
-
24 Nov 2021 1 repository listed
-
13 Jul 2021 1 repository listed
-
27 Feb 2019 1 repository listed
-
1 Jan 2019 1 repository listed
-
Semivalue-based data valuation is arbitrary and gameable14 Jun 2025 0 repositories listed
-
Fast-DataShapley: Neural Modeling for Training Data Valuation5 Jun 2025 0 repositories listed
-
Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value22 May 2025 0 repositories listed
-
Efficient Data Valuation Approximation in Federated Learning: A Sampling-based Approach23 Apr 2025 0 repositories listed
-
From Fairness to Truthfulness: Rethinking Data Valuation Design7 Apr 2025 0 repositories listed
-
Monte Carlo Sampling for Analyzing In-Context Examples27 Mar 2025 0 repositories listed
-
LLM-Aided Customizable Profiling of Code Data Based On Programming Language Concepts19 Mar 2025 0 repositories listed
-
Optimizing Product Provenance Verification using Data Valuation Methods21 Feb 2025 0 repositories listed
-
LiveVal: Time-aware Data Valuation via Adaptive Reference Points14 Feb 2025 0 repositories listed
-
On the Impact of the Utility in Semivalue-based Data Valuation10 Feb 2025 0 repositories listed
-
Unifying and Optimizing Data Values for Selection via Sequential-Decision-Making6 Feb 2025 0 repositories listed
-
Data Overvaluation Attack and Truthful Data Valuation in Federated Learning1 Feb 2025 0 repositories listed
-
Fairshare Data Pricing via Data Valuation for Large Language Models31 Jan 2025 0 repositories listed
-
Fortifying Federated Learning Towards Trustworthiness via Auditable Data Valuation and Verifiable Client Contribution1 Jan 2025 0 repositories listed
-
Data value estimation on private gradients22 Dec 2024 0 repositories listed
-
Data Acquisition for Improving Model Fairness using Reinforcement Learning4 Dec 2024 0 repositories listed
-
Dissecting Representation Misalignment in Contrastive Learning via Influence Function18 Nov 2024 0 repositories listed
-
WaKA: Data Attribution using K-Nearest Neighbors and Membership Privacy Principles2 Nov 2024 0 repositories listed
-
Private, Augmentation-Robust and Task-Agnostic Data Valuation Approach for Data Marketplace1 Nov 2024 0 repositories listed
-
Investigating Layer Importance in Large Language Models22 Sep 2024 0 repositories listed
-
Exploiting the Data Gap: Utilizing Non-ignorable Missingness to Manipulate Model Learning6 Sep 2024 0 repositories listed
-
Reframing Data Value for Large Language Models Through the Lens of Plausibility30 Aug 2024 0 repositories listed
-
Disentangled Structural and Featural Representation for Task-Agnostic Graph Valuation22 Aug 2024 0 repositories listed
-
Uncertainty Quantification of Data Shapley via Statistical Inference28 Jul 2024 0 repositories listed
-
Personalization of Dataset Retrieval Results using a Metadata-based Data Valuation Method22 Jul 2024 0 repositories listed
-
Towards Understanding the Influence of Training Samples on Explanations5 Jun 2024 0 repositories listed
-
Is Data Valuation Learnable and Interpretable?3 Jun 2024 0 repositories listed
-
Proper Dataset Valuation by Pointwise Mutual Information28 May 2024 0 repositories listed
-
Data Valuation by Leveraging Global and Local Statistical Information23 May 2024 0 repositories listed
-
Rethinking Data Shapley for Data Selection Tasks: Misleads and Merits6 May 2024 0 repositories listed
-
Neural Dynamic Data Valuation30 Apr 2024 0 repositories listed
-
Incentives in Private Collaborative Machine Learning2 Apr 2024 0 repositories listed
-
DAVED: Data Acquisition via Experimental Design for Data Markets20 Mar 2024 0 repositories listed
-
VTruST: Controllable value function based subset selection for Data-Centric Trustworthy AI8 Mar 2024 0 repositories listed
-
EcoVal: An Efficient Data Valuation Framework for Machine Learning14 Feb 2024 0 repositories listed
-
Efficient Data Shapley for Weighted Nearest Neighbor Algorithms20 Jan 2024 0 repositories listed
-
Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective9 Jan 2024 0 repositories listed
-
Load Data Valuation in Multi-Energy Systems: An End-to-End Approach16 Nov 2023 0 repositories listed
-
Threshold KNN-Shapley: A Linear-Time and Privacy-Friendly Approach to Data Valuation30 Aug 2023 0 repositories listed
-
Shapley Value on Probabilistic Classifiers12 Jun 2023 0 repositories listed
-
Cooperative IoT Data Sharing with Heterogeneity of Participants Based on Electricity Retail31 May 2023 0 repositories listed
-
New methods for new data? An overview and illustration of quantitative inductive methods for HRM research15 May 2023 0 repositories listed
-
GMValuator: Similarity-based Data Valuation for Generative Models21 Apr 2023 0 repositories listed
-
Optimizing Data Shapley Interaction Calculation from O(2^n) to O(t n^2) for KNN models2 Apr 2023 0 repositories listed
-
Fairness-Aware Data Valuation for Supervised Learning29 Mar 2023 0 repositories listed
-
A Note on "Towards Efficient Data Valuation Based on the Shapley Value''22 Feb 2023 0 repositories listed
-
IPProtect: protecting the intellectual property of visual datasets during data valuation22 Dec 2022 0 repositories listed
Syntology lines on 21 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections