Candidate Retention for Abductive Learning

Classical ABL selects one valid candidate and uses it as supervision (Dai et al.,, 2019). It commonly ranks candidates by confidence or Hamming distance. Under the factorized perception model, candidate confidence is

A Hamming score measures the negative distance between ${\bm{c}}$ and the predicted concept sequence. Writing either score generically as $\operatorname{score}_{\theta}$, ABL selects

The selected candidate provides hard labels for updating $f_{\theta}$. The updated model then reranks the candidates.

$\text{A}^{3}\text{BL}$ keeps all valid candidates and weights them by confidence (He et al., 2024b, ). Their total confidence is

Normalizing by $Z_{\theta}({\bm{x}},y)$ gives the candidate posterior

We assume ${\bm{s}}({\bm{x}},y)\neq\varnothing$ and $Z_{\theta}({\bm{x}},y)>0$.

$\text{A}^{3}\text{BL}$ marginalizes this posterior at each sequence position to construct soft concept targets for $f_{\theta}$.

Classical ABL and $\text{A}^{3}\text{BL}$ provide the single-candidate and posterior-marginalization reference policies. We next consider how the retained subset shapes concept supervision and use this information to guide selection.

For a training pair, let $R\subseteq{\bm{s}}({\bm{x}},y)$ be the retained candidate set. Conditioning the candidate distribution on $R$ gives its retained mass and conditional distribution:

where $M_{\theta}(R;{\bm{x}},y)>0$.

At position $j$, the marginal of $q_{\theta,R}$ is

Retained uncertainty averages one minus the largest marginal probability across positions:

A lower $\mathrm{U}_{\theta}(R;{\bm{x}},y)$ gives sharper concept targets. The discarded posterior mass is $1-M_{\theta}(R;{\bm{x}},y)$.

To relate the retained supervision to concept error, let $P^{\star}(\cdot\mid{\bm{x}},y)$ be the unknown true distribution over valid candidates. It is a point mass when each input sequence has one true candidate.

The retained prediction at position $j$ is

Its ideal risk is the expected coordinate error under $P^{\star}$:

Since $P^{\star}$ is unknown, we compare the induced supervision with this distribution through the following upper bound.

At round $t$, the current parameters $\theta_{t}$ and retained set $R_{t}$ define soft targets $\pi_{t,j}(k)=\pi_{\theta_{t},R_{t},j}(k)$ and mass $M_{t}=M_{\theta_{t}}(R_{t};{\bm{x}},y)$. Holding these quantities fixed during the neural update gives the normalized cross-entropy loss

Marginalization pools label support across joint explanations. Several candidates can agree at one position while assigning different labels elsewhere; their weights add to the same target entry at that position. Every input in a sequence receives a target derived from the same retained distribution. A singleton produces one-hot targets, while retaining the full valid set recovers its full-distribution marginals. Intermediate sets preserve the alternatives that contribute to the selected supervision.

In aggregated mod addition, the implementation uses the original retained weights to form the targets:

Its loss is $M_{t}{\mathcal{L}}_{\mathrm{ret}}(\theta;\pi_{t})$, with $M_{t}$ also fixed during the update. The conditional marginals specify the relative label distribution, and retained mass weights the training example. Appendix B gives the scoring and weighting conventions. The standard EM connection for fixed sets, product-posterior weights, and normalized targets is described in Appendix A.1.

We construct $R\subseteq C_{K}$ by forward selection, starting with the highest-weight candidate. The set expands only when adding a candidate lowers the criterion.

To see how a candidate changes supervision, write $M=M_{\theta}(R;{\bm{x}},y)$, $w_{{\bm{c}}}=w_{\theta}({\bm{c}}\mid{\bm{x}},y)$, and $\pi_{j}=\pi_{\theta,R,j}$. Adding ${\bm{c}}\notin R$ yields marginals $\pi_{j}^{+}=\pi_{\theta,R\cup\{{\bm{c}}\},j}$ with

The candidate contributes mass to its assigned label at each position. The effect on uncertainty depends on where this support falls relative to the existing label masses. Two candidates with equal weights can therefore produce different changes in uncertainty. A weight-only top-$k$ rule ranks them independently of the current retained set. ACR evaluates their effects on the supervision already formed by that set, using the following change in the criterion.

For ${\bm{c}}\in C_{K}\setminus R$, define

The first difference measures the change in retained ambiguity. The final term rewards the posterior mass recovered by ${\bm{c}}$.

At each step, we select the candidate with the smallest $\Delta_{{\bm{c}}}J_{\theta}(R)$. We add it if this value is negative and stop otherwise.

Each accepted step strictly lowers $J_{\theta}$ and adds one candidate, so the algorithm terminates at a one-addition local optimum over $C_{K}$. With maintained coordinate label masses, selection takes $O(N^{2}m)$ time in the worst case, where $N=|C_{K}|\leq K$. At round $t$, the selected $R_{t}$ defines the fixed targets in Section 3.2. Candidate weights and retention are recomputed after the neural update.

We begin with two arithmetic settings where the same observed target admits multiple valid digit assignments.

In digit addition, each example contains two sequences of $n$ digit images whose hidden labels form two integers; training observes only their sum (Manhaeve et al.,, 2018). For $n=1$, SUM(,)=3 reveals the sum 3, while the image labels remain hidden. MNIST (LeCun et al.,, 1998) and KMNIST (Clanuwat et al.,, 2018) supply the images for $n\in\{1,2,3,4\}$. We report character accuracy on the recovered digit labels, averaged over five runs. As $n$ grows, more hidden decisions must agree with the same target.

For these arithmetic tasks, we compare ACR with the single-candidate ABL-hamming and ABL-confidence (Dai et al.,, 2019), and $\text{A}^{3}\text{BL}$  (He et al., 2024b, ). Digit addition also includes DeepProbLog, NeurASP, and DeepStochLog (Manhaeve et al.,, 2018; Yang et al.,, 2020; Winters et al.,, 2022). For aggregated mod addition, the $\text{A}^{3}\text{BL}$ runs use a top-32 cap and temperature $0.2$, retaining all candidates when at most 32 are available.

Both ACR and $\text{A}^{3}\text{BL}$ maintain high character accuracy across operand lengths, whereas the hard-selection methods have lower means or larger variation across runs (Table 1). On KMNIST, ACR exceeds $\text{A}^{3}\text{BL}$ for $n\geq 2$, with gains of 0.81–1.38 percentage points; $\text{A}^{3}\text{BL}$ performs better at $n=1$.

Digit addition tests scaling in the number of hidden positions but uses a single rule per example. Aggregated mod addition introduces a harder coupling: a shared digit predictor must satisfy two modular rules simultaneously. For a modulus $k$, each example contains two one-digit images with hidden labels $z_{1},z_{2}$ and one observed residue:

Each training example samples $k\in\{k_{1},k_{2}\}$ and all examples share one digit predictor, so MODSUM_3(,)=0 and MODSUM_4(,)=3 follow different modular rules within the same training set. We evaluate 19 modulus pairs with 30,000 examples and three runs per pair (He and Li,, 2025).

The representative curves in Figure 2 reveal a difference in learning dynamics: ACR maintains high accuracy after the initial rise, while $\text{A}^{3}\text{BL}$ declines after an earlier peak in some settings. Across the evaluated modulus pairs, ACR improves concept accuracy over $\text{A}^{3}\text{BL}$ in most settings. The gains vary across pairs; Appendix C reports the complete results, including small or negative gains.

The retention objective $J_{\theta}$ balances retained uncertainty against discarded posterior mass. We study these roles through controlled ablations of the two terms and a training trace of the resulting supervision. Both use retained true-label weight (TLM) as a diagnostic. Let $z^{\star}=(z_{j}^{\star})_{j=1}^{m}$ denote the ground-truth concepts; we define

Here $\widetilde{w}_{\theta}$ is the candidate distribution used in the experiments (Appendix B). TLM averages the retained weight assigned to the correct concept without renormalizing over $R$; it therefore reflects both retained mass and label quality.

The ablations in Table 2 compare three retention policies under a shared training protocol. Without the mass term, the initial singleton already minimizes uncertainty, $\mathrm{U}_{\theta}=0$, so the strict-improvement rule accepts no additions. Without uncertainty, each positive-weight addition reduces discarded mass, and the rule retains the full search pool of $\min(32,|{\bm{s}}|)$ candidates. Both controls use the same scoring and raw-weight supervision as ACR.

Averaged over the aggregated mod-addition settings, ACR reaches 94.98% concept accuracy, compared with 20.01% for ACR w/o mass and 65.22% for ACR w/o uncertainty (Table 2). Removing either term also lowers retained true-label weight. ACR retains fewer candidates on average than ACR w/o uncertainty while assigning more weight to correct labels. Figure 3 shows how these differences vary across modulus pairs.

The representative trace in Figure 4 follows the retained supervision on MNIST mod addition $(3,4)$. Retained uncertainty rises early and then falls, while discarded mass approaches zero. In this trace, sharper coordinate targets coexist later in training with coverage of nearly all model-posterior mass.

We next test ACR on two non-arithmetic tasks: chess attack and judicial sentencing, comparing with ABL-hamming, ABL-confidence (Dai et al.,, 2019), and $\text{A}^{3}\text{BL}$  (He et al., 2024b, ). In chess attack, MNIST images occupy board squares and map to six piece types (Dai et al.,, 2019). Training provides square locations and whether any pair attacks; piece labels remain hidden. We report attack and piece accuracy from a single run on about 60,000 training and 10,000 test boards. All methods share the split and 20-epoch schedule; ACR uses five warm-up epochs of $\text{A}^{3}\text{BL}$ supervision before switching to retention-based selection.

Judicial sentencing takes judgment facts as input and predicts discrete sentencing factors, which a knowledge base maps to the observed prison term (Huang et al.,, 2020). We follow the task setup of Huang et al. and report Macro-F1 and Micro-F1, averaged over five runs.

Warm-started ACR improves more on piece identification than on attack prediction (Table 3). Both marginalization-based methods outperform hard selection, while the judicial task leaves less room for adaptive retention to differentiate from $\text{A}^{3}\text{BL}$ .

Neuro-symbolic methods connect perception and reasoning through architectures, loss functions, or explicit inference. Early systems embedded declarative rules in connectionist architectures (Towell and Shavlik,, 1994; d’Avila Garcez et al.,, 2002). Logic-as-loss methods translate symbolic constraints into differentiable objectives (Xu et al.,, 2018; Roychowdhury et al.,, 2021); their behavior depends on the chosen relaxation (van Krieken et al.,, 2022; He et al., 2024a, ).

Hybrid systems retain an explicit symbolic reasoner. DeepProbLog extends ProbLog with neural predicates (Manhaeve et al.,, 2018; Manhaeve et al.,, 2021), NeurASP couples neural perception with answer-set programming (Yang et al.,, 2020), and DeepStochLog uses stochastic logic programs (Winters et al.,, 2022). They share a modular connection between neural perception and symbolic reasoning, with different mechanisms for exchanging information. In ABL, abduction returns explanations that become labels for the neural update. This feedback interface makes the candidate-to-supervision step explicit: several valid joint assignments can supply different labels for the same inputs. ACR studies which of these explanations should contribute to the returned supervision and how their labels combine into concept targets.

ABL turns abduction, inferring hypotheses that explain observations under background knowledge (Magnani,, 2009), into a learning mechanism whose central aim is to make perception and reasoning mutually beneficial (Zhou,, 2019; Dai et al.,, 2019). The neural model predicts concepts, abduction revises them under the knowledge base, and the revised candidates supervise the next neural update. When several candidates explain the same target, classical ABL selects the one closest to the current prediction, using Hamming distance or model confidence. This decision depends on model quality: at cold start, an unreliable prediction can select a poor pseudo-label, which subsequent updates may reinforce (Tao et al.,, 2024; He et al., 2024b, ). ABLSim mitigates this dependence with inter-sample similarity that guides consistency optimization before the classifier becomes reliable (Huang et al.,, 2021).

Subsequent work extends the loop along three axes. (i) Knowledge scope: GABL enables abduction from inexact ground propositions (Cai et al.,, 2021), and related methods learn from raw data, semi-supervised signals, or inaccurate rules (Dai and Muggleton,, 2021; Huang et al.,, 2020; Yang et al.,, 2024). The framework has also been applied to PDE discovery (Gao et al.,, 2025) and document recognition (Gao et al.,, 2024). (ii) Identifiability: formal analyses connect learning reliability to knowledge-base structure (Tao et al.,, 2024; He and Li,, 2025). (iii) Reasoning control: C-ABL partitions the knowledge base into sub-bases introduced progressively (Hu et al.,, 2025).

These advances improve what knowledge is available or how reasoning is structured, but leave a separate design choice: how much of the candidate posterior should return to the neural model as supervision. This choice matters because target-level consistency need not identify the intended latent concepts: different candidates can explain one target while assigning different concept labels, producing reasoning shortcuts (Marconato et al.,, 2023; Bortolotti et al.,, 2024). $\text{A}^{3}\text{BL}$ addresses this ambiguity by weighting every valid candidate under the current model and marginalizing their coordinate labels (He et al., 2024b, ), avoiding an early single-candidate commitment. Classical ABL and $\text{A}^{3}\text{BL}$ therefore define the single-candidate and marginalization endpoints. ACR develops the intermediate range through a greedy subset-selection criterion combining supervision sharpness and posterior-mass coverage.