NBA 2014–15 shot logs · checks on the published model

Model Checks

Two exploratory checks on the published model. One asks whether a scenario estimate depends heavily on the logit form. The other maps conditional dependencies among the same factors under a different structural representation.

The published model remains the reference. Neither check replaces it or becomes a second reference model.

Triangulation Check

How dependent is the estimate on model form?

Model: propensity subclassification + Bayesian network

Select a scenario by setting one or more factors. The check asks whether it produces a similar result through two structurally different routes: propensity-score subclassification, and the fitted Bayesian network where it can represent the scenario.

  • Agreement narrows one concern: that the result is an artifact of the published logit specification.
  • Material disagreement points the other way: the scenario should be described as model-form dependent rather than presented as though the alternative route supported it.
More detail

Where a scenario factor reaches the outcome only through another factor, the network reports a projection that lets the other factors respond. It is labeled a different estimand and is comparable on direction rather than size. Where a factor has no path to the outcome at all, the check reports no network estimate and refers the reader to the network view instead of showing a zero.

The verdict is built from the model and propensity routes, so the network can lower the verdict but never raise it.

The routes use the same predictors and the same sample. The check does not address omitted measurement, nonresponse, causal direction, or other shared vulnerabilities.

The check applies Mosteller’s idea of triangulation: a difficult quantity is estimated through several structurally different models.

Bayesian Network Analysis

BETA

How are the selected factors conditionally related?

Model: constrained Bayesian network

Maps conditional dependencies among the published model’s factors under a different structural representation. It shows which factors have a path to the outcome in the fitted network.

  • A factor with a path to the outcome is a direct antecedent of it, or reaches it only through another factor.
  • A factor with no path may still add information in the logit; the two structures can place the same measured factor differently.
More detail

The what-if view reports each scenario two ways: a projection that lets the other factors respond within the fitted network, and a figure that holds them at each record’s own values, the rule the published model’s scenarios use. A factor that reaches the outcome only through another factor moves the first figure but not the second.

The result remains conditional on the network’s structural assumptions. Neither the learned arrows nor the what-if estimates establish causal direction, and the view is not a second reference model.

Narrowing to a subgroup resolves that subgroup’s own predictor set. That answers a different question — what explains the outcome within that group — rather than whether the published explanation survives a change of structure. To refit the published specification on a subgroup, build it in the Model Builder; the Triangulation Check then runs within that filter.

How they work together

One probes model-form dependence; the other maps conditional dependencies.

Triangulation probes model-form dependence

Does a scenario estimate survive a change of structure? Similar outcomes from differing structures are themselves evidence.

The network maps conditional dependencies

Which factors are direct antecedents of the outcome, and which reach it only through another factor? The arrows describe the fitted network and do not establish causal direction.

A factor can matter in the logit and have no path in the network. Structure selection and functional form vary together in any comparison with the logit, so read the comparison as evidence about direction rather than magnitude.

Agreement between the Bayesian-network and logit results is evidence about model-form dependence rather than about cause. Neither check establishes causal direction or addresses omitted measurement, nonresponse, or other shared vulnerabilities.