Developing a Pneumonia Model as a Learning Project

A hobby project about image classification, transfer learning, and the evaluation of neural networks

I started this project because I wanted to understand how a neural network for images is built and trained. I wanted to work through the entire process myself: define the architecture, split the data, choose a loss function, run the training, and evaluate the results. I was particularly interested in how individual design decisions and seemingly incidental properties of the images would affect the model.

The goal was explicitly not to achieve the highest possible sensitivity or specificity for pneumonia detection. A hobby project is neither the right setting nor the right data foundation for that. Medical image classification served as a concrete task through which I could follow and test each step. Metrics such as AUC and calibration error helped me compare variants, but they were not the purpose of the project.

The final model is a ResNet-18 with two outputs. The first estimates the probability of an opacity consistent with pneumonia. The second produces a spatial map that points towards a possible location of that opacity. Five separately trained models are later combined into an ensemble.

Medical disclaimer: This project is a research demonstrator, not a medical device. It must not be used for diagnosis or treatment decisions.

Model Architecture

The model processes a frontal chest radiograph and answers two questions:

1. How likely is an opacity consistent with pneumonia?
2. Where in the image might it be located?

Both tasks initially share the same ResNet-18 backbone. Only in the later layers does the network split into a classification head and a localisation head.

The small images in Figure 1 are not reduced copies of the radiograph. Each one shows a single channel from the respective layer. A channel responds to a pattern that became useful during training. In most cases, it cannot simply be labelled a “rib channel” or a “pneumonia channel.”

In the stem, the image is first convolved, normalised, and passed through a ReLU function. Max pooling then reduces the spatial resolution to 56 × 56. Edges and larger anatomical shapes remain recognisable in the early maps. Layer 1 retains the resolution and combines these simple structures. Layers 2 and 3 reduce the grid to 28 × 28 and 14 × 14. The activations become coarser and increasingly concentrated on individual regions.

After layer 3, the localisation head uses the 256 channels to calculate one location score for each tile in the 14 × 14 grid. The other branch is compressed further in layer 4, down to 7 × 7 positions. Only then are the spatial values averaged and translated into a single classification score.

Starting Model and Transfer Learning

The starting point is ResNet-18 from the ResNet paper by He et al.. Training begins with Torchvision's IMAGENET1K_V1 weights.

Those weights come from training on photographs from 1,000 ImageNet classes. The model therefore has no prior knowledge of pneumonia. It can, however, already process elementary visual structures such as edges, curves, surfaces, and shapes. Transfer learning reuses this knowledge as the starting point for a new task.

The original output layer for the 1,000 ImageNet classes is removed and replaced by the two project-specific heads. All layers of ResNet-18 are then trained further on the RSNA radiographs. This complete adaptation of the pretrained network is the fine-tuning stage.

Image Preprocessing and Shared Backbone

The code reads each radiograph as a greyscale image, resizes it to 224 × 224 pixels, and copies it into three channels because ResNet-18 expects three input channels. It then applies the fixed means and standard deviations used for ImageNet normalisation.

Using fixed constants was a deliberate decision. An earlier variant normalised each radiograph independently. That initially sounded plausible, but in the experiment it strengthened the unwanted cue indicating the acquisition projection.

Within ResNet-18, spatial resolution decreases while the number of feature maps increases:

Stage	Spatial size	Channels	Role in the model
Input	224 × 224	3	Greyscale image copied into three channels
Stem / layer 1	56 × 56	64	Edges and local textures
Layer 2	28 × 28	128	Larger patterns
Layer 3	14 × 14	256	Regional features
Layer 4	7 × 7	512	Highly compressed features

The “Res” in ResNet stands for residual. A residual block calculates a correction to its input and adds the two together. These short connections help information and learning signals move through the network.

Classification Head

The classification head processes the output of layer 4. At this point there are 7 × 7 positions with 512 features each. The model averages across the 49 positions, producing a vector of 512 numbers. A linear layer compresses that vector into a single logit.

A logit is a raw score without an immediately interpretable probability scale. A sigmoid function maps it to the range from 0 to 1. The resulting value already looks like a probability, but it is not necessarily a reliable one. A model can rank cases in the correct order while still returning systematically high or low numbers. Platt calibration later corrects this scale.

Localisation Head

The localisation head branches off after layer 3, where the network still has a 14 × 14 grid representing 196 image regions. A single 1 × 1 convolution translates the 256 features of each tile into a location score.

The shallow design of this head is part of the experimental setup. A more elaborate head would have changed both the localisation supervision and the model capacity at the same time. With a single 1 × 1 convolution, it is easier to assess the effect of the additional spatial information itself.

The localisation head is trained using rectangles drawn by radiologists. It therefore learns directly which regions contain an opacity.

Grad-CAM is different. Its map is calculated only after prediction, using the classification branch. It shows which regions in the final convolutional block influenced the classification score. Grad-CAM was not trained on the radiologists' boxes.

The localisation head contains 14 × 14 values, while Grad-CAM contains only 7 × 7. Both maps are enlarged and smoothed for display. This does not create any additional image information.

Data and Target Variable

The main dataset is the RSNA Pneumonia Detection Challenge on Kaggle. In this project, it contains 26,684 frontal chest radiographs divided into three classes:

Class	Meaning
Lung Opacity	Possible opacity consistent with pneumonia; positive cases include rectangular annotations
Normal	No abnormal finding
No Lung Opacity / Not Normal	Abnormal, but without the target opacity

For the binary model, Lung Opacity = 1. The other two classes are treated as 0.

The third class prevents the task from collapsing into “diseased versus healthy.” Almost half of the AP images belong to this abnormal non-pneumonia group. Without it, the model could even more easily equate a bedside acquisition with pneumonia.

The source data are converted from DICOM to PNG files measuring 512 × 512 pixels. DICOM is a medical image format that can store information such as the acquisition projection and pixel spacing in addition to the image pixels. The conversion changes neither contrast nor crop. Those operations remain in the training code so that they can be tested individually.

Other public datasets serve clearly separated purposes:

Dataset	Use in this project	Link
RSNA Pneumonia Detection Challenge	Training, cross-validation, and internal holdout evaluation of the classifier	Kaggle · Dataset paper
Kermany Chest X-Ray	External evaluation; previously rejected as a training source because of an image-size leak	Mendeley Data · Paper
VinDr-CXR	External evaluation in adults; classification and localisation	PhysioNet · Paper
Montgomery County and Shenzhen	Training the separate U-Net lung finder	NIH/NLM dataset page

The six included demo images come from the RSNA/NIH collection. Their reproducible selection is documented in manifest.json. Provenance and licence information are provided in NOTICE.md.

Training and Evaluation Design

Data Splitting and Balancing the Training Stream

At the beginning of the project, 3,812 images—approximately 15 per cent of the dataset—were locked away as a holdout set. This part remained untouched during development and was evaluated only after the model decisions had been finalised.

The remaining 22,872 images form the development set. They are divided into five folds at patient level. During five-fold cross-validation, each model is trained on four folds and evaluated on the fifth. After five runs, every development image has been evaluated exactly once by a model that did not use it for training.

The split is additionally stratified by diagnosis and acquisition projection. AP and PA images, as well as positive and negative cases, therefore remain similarly distributed across all partitions.

This stratification makes the folds comparable, but it does not remove the association between acquisition projection and diagnosis. The training stream is therefore balanced as an additional step. Underrepresented combinations of projection and diagnosis are sampled more often, while overrepresented combinations are sampled less often. “Filling up” does not mean generating new or artificial radiographs. Existing images from the rarer groups simply appear more frequently during training. The balancing applies only to the respective training partition. The selection split, outer evaluation fold, and holdout retain their original distributions.

This balancing is necessary because the model can otherwise use the acquisition projection as a confounder. Of all the measures tested in the project, weighted sampling was the only one that measurably reduced the influence of projection. The section on controlling the projection confounder describes the weighting and its effect in more detail.

Each training partition also contains a separate selection split. It determines the best training epoch, the calibration, and the later decision threshold. The outer evaluation fold remains untouched by these decisions.

Data Augmentation

During training, each image is modified slightly:

• rotation of up to 7 degrees
• translation of up to 3 per cent
• scaling between 0.93 and 1.07
• brightness and contrast variation of 0.15

The purpose of augmentation is to stop the network from memorising individual pixel patterns. Horizontal flipping is not used because it would reverse left and right, mirror the cardiac silhouette and side markers, and create anatomically incorrect examples.

For geometric transformations, the image and the radiologist's rectangle must receive exactly the same transformation. Two independent random movements would separate the location label from the opacity.

Loss Function

During training, the correct answer is known for every image. The model might, for example, return a pneumonia probability of 0.70 while the stored class is either 1 for pneumonia or 0 for no pneumonia. To update the weights, this discrepancy has to be translated into a number. That number is the loss.

A simple “right” or “wrong” would be too coarse. For a positive image, both 0.51 and 0.99 would be formally correct decisions, but the second prediction fits the target class much better. Conversely, a confident error of 0.99 on a negative image should count more heavily than an uncertain value of 0.51.

After every training batch, the code calculates how a small change in each model weight would affect the loss. This backward calculation is called backpropagation. The optimiser then changes the weights in the direction that reduces the loss. A loss function is therefore not an additional evaluation metric for the finished classifier. It supplies the signal that allows the model to learn in the first place.

Binary cross-entropy is commonly used when there are two possible classes. “Binary” refers to the two target values 0 and 1. Its formula is:

BCE = -(y × ln(p) + (1 - y) × ln(1 - p))

Here, y is the correct class and p is the predicted probability of the positive class. ln denotes the natural logarithm. Because y is either 0 or 1, one half of the formula disappears in each case:

• For a positive image with y = 1, the expression becomes -ln(p). The closer p is to 1, the smaller the loss.
• For a negative image with y = 0, the expression becomes -ln(1 - p). The closer p is to 0, the smaller the loss.

The behaviour can be summarised without calculating logarithms:

Correct class	Model score p	Cross-entropy assessment
Positive	0.90	Small loss
Positive	0.10	Large loss
Negative	0.10	Small loss
Negative	0.90	Large loss

Highly confident errors are penalised particularly strongly. This is intentional: a model that is almost certain about the wrong answer needs a stronger correction signal than one that is still undecided.

The project uses binary cross-entropy in two places. The classification head calculates one loss per image. Positive cases are less common and receive a higher weight through pos_weight, so the more numerous negative cases do not dominate training. The localisation head calculates the same type of error for every tile in its 14 × 14 location field.

The total loss is:

total loss = classification loss + λ × localisation loss

The value of lambda, λ, was not chosen by intuition. In the first training batch containing an annotated opacity, the code measures both losses and selects lambda so that their initial magnitudes are similar. The value then remains fixed.

Only images with a radiologist's box contribute to the localisation loss in the deployed model. The head therefore learns a conditional task: if pneumonia is present, where is it? It does not reliably learn to return low values everywhere on normal images. The application consequently displays the location field as a soft indication, without a box or a hard threshold.

Controlling the Projection Confounder

Acquisition projection proved to be the most important confounder in the project. AP images are often acquired with mobile equipment at the bedside and therefore frequently involve patients who are more severely ill. PA images are more commonly acquired while the patient is standing at a wall-mounted detector. In the development set, 38.3 per cent of AP images but only 9.3 per cent of PA images contain pneumonia. Projection alone predicts the diagnosis with an AUC of 0.706.

The network can exploit this association even though projection is not the target finding. Several attempts to remove the projection cue from the pixels were not sufficiently successful. Weighted sampling, already described in the data-splitting section, was the only measure that measurably reduced the confounder.

Images are sampled so that projection and diagnosis are statistically independent in the training stream. The overall AP-to-PA ratio remains unchanged, as does the total number of positive cases. Only the link between both properties drops from an AUC of 0.706 to 0.500.

The weights follow this formula:

w(projection, diagnosis) = expected frequency under independence / observed frequency

This measure reduced the projection channel more than any other intervention, but it also cost diagnostic performance. Compared with the baseline model, the projection channel fell by 0.0554 AUC while the stratified diagnostic AUC fell by 0.0181.

Training Configuration

Each of the five runs uses the same configuration:

• ResNet-18 with ImageNet initial weights and two heads
• input size of 224 × 224 pixels and batch size of 16
• eight epochs
• AdamW with a learning rate of 3e-4 and weight decay of 1e-4
• one-cycle learning-rate schedule
• weighted sampling to decouple acquisition projection and diagnosis
• checkpoint selection using the inner selection split

The optimiser uses the loss to determine how the model weights should change. The learning-rate schedule controls the size of those updates during training. The complete final recipe is stored in train_final_model.ps1.

The eight epochs did not develop identically across the five folds. Classification loss on the training images decreased continuously. On the inner selection split, its mean reached its minimum in epoch 2 and then increased again. At the same time, mean AUC on the same selection split continued to improve slightly. This is not a contradiction: AUC considers only the ranking of cases, whereas binary cross-entropy also evaluates how well the numerical predictions fit. A ranking can therefore improve while the probability scale becomes worse.

Ensemble and Calibration

Five independent models remain after cross-validation. For each prediction, all five evaluate the same image. Before their outputs are averaged, each model is calibrated separately using Platt calibration.

The five paths in Figure 3 therefore represent five different parameter sets, not five repetitions of the same model. Fold 0 returns a raw score that is corrected by fold 0's Platt curve; the same happens for folds 1 to 4 using their respective curves. Only the five calibrated probabilities are averaged into the ensemble output.

Why Calibration Is Necessary

A sigmoid score of 0.70 initially means only that the model returned that number. It would be a well-calibrated probability only if, among many comparable cases with predictions around 0.70, approximately 70 per cent were actually positive.

Training does not guarantee this correspondence. In this project, positive cases receive more weight through pos_weight so that they are not overwhelmed by the more common negative cases in the loss. This weighting helps the model learn class separation, but it shifts the probability scale. Across the 22,872 development images, the mean raw prediction was 0.334 even though only 0.225 of the images were positive. On average, the model therefore overstated the risk.

One way to think about this is a thermometer with a mislabelled scale. It can reliably order warmer and colder objects, but it systematically reports temperatures that are too high. Calibration corrects the scale labels; it does not alter the trained ResNet-18 itself.

How the Platt Curve Works

Platt calibration learns a simple S-shaped curve with two parameters. It receives the existing model score p and returns a corrected probability:

p_calibrated = Sigmoid(a × Logit(p) + b)

Parameter b shifts the scale up or down. Parameter a changes its steepness. If a model is overconfident, the curve can pull extreme values towards the centre. Both parameters are learned from examples for which the model prediction and correct class are known.

Each of the five models receives its own curve, fitted on its inner selection split. The respective model did not use these images to learn its ResNet weights. Neither the outer evaluation fold nor the holdout is used to fit the curve.

The Platt curve is monotonic. The order of cases within one model therefore remains unchanged: an image with a higher raw score still has the higher score after calibration. The curve does not retrospectively make the classifier better at ranking positive and negative cases. Its purpose is to make the reported number more meaningful as a probability.

Calibration and the decision threshold are separate. Calibration corrects the meaning of the entire probability scale. The threshold subsequently determines the calibrated value above which the system labels a case positive.

The calculation has three steps:

1. convert each model's raw score using sigmoid
2. calibrate each model with its own Platt curve
3. average the five calibrated probabilities

After calibration, the mean prediction on the development set was 0.2255, close to the actual positive fraction of 0.225. On the holdout, expected calibration error fell from 0.1029 to 0.0260. Only after this correction are the five probabilities averaged into the ensemble score.

The web application also evaluates five minimally altered crops of the image. Their range indicates how sensitive the prediction is to a two-per-cent change in framing. This range is a stability check, not a statistical confidence interval. The model assessment continues to use the unchanged full image.

Evaluation Results

The holdout was evaluated once, after development had been completed.

Metric	Result
Stratified AUC of the ensemble	0.8687 [0.8566, 0.8805]
Mean stratified AUC of individual models	0.8473
Localisation head, point AUC within the lungs	0.9123
Grad-CAM, point AUC within the lungs	0.7312
Fixed anatomical position template	0.7520
Calibration error after Platt calibration	0.0260
Model score reveals AP versus PA	0.7501

Here, AUC describes how often a randomly selected positive case receives a higher model score than a randomly selected negative case. A value of 1.0 would be perfect, while 0.5 would correspond to a coin toss.

Diagnostic AUC is calculated separately for AP and PA images and then combined. This stratification prevents the model from benefiting solely from the different pneumonia prevalence of the two acquisition projections.

The localisation head's point AUC is higher than that of both Grad-CAM and a fixed position template. This template uses the same average location of all training rectangles for every image and never looks at the current radiograph. Grad-CAM performs less well because it was not trained on the radiologists' boxes and explains a different quantity.

The localisation head's point AUC was calculated by cross-validation across all 5,154 positive development images. Each image was evaluated by a model that had not used it for training.

Figure 4 shows that the two maps respond differently. In the positive case, the trained localisation head focuses clearly on the opacity. Grad-CAM partly emphasises different regions. Activations also appear in the normal and abnormal negative cases because the localisation head receives a spatial loss only for positive images.

The final row of the results table limits the interpretation of the diagnostic performance. Acquisition projection can still be reconstructed from the model score with an AUC of 0.7501. Weighted sampling reduced this channel, but did not remove it.

External Validation

The holdout evaluation uses withheld cases, but they still come from the same data source as the training images. For a stricter test, I therefore applied the unchanged ensemble to two external datasets: first to paediatric images and then to images from adults. The weights were not fine-tuned, and neither calibration nor the decision threshold was adapted to the new data.

Paediatric Chest Radiographs: Kermany Dataset

The Kermany dataset contains 5,856 anteroposterior chest radiographs from children aged one to five years, acquired at Guangzhou Women and Children's Medical Center. Pneumonia is present in 4,273 images, while 1,583 are considered normal. The resulting prevalence of 73.0 per cent is substantially higher than in the RSNA training data. Age, origin, file format, and image acquisition also differ from the training data. This evaluation therefore tests more than unseen images: it represents a considerable shift in both population and technical domain. The dataset's medical provenance and construction are described in the original publication.

Metric	Result
Images	5,856
Patient groups	3,054
Pneumonia prevalence	73.0%
Ensemble AUC	0.934 (95% CI: 0.928–0.941)
AUC after controlling for the image-size confounder	0.923
ECE of the unchanged calibration	0.478
Sensitivity at the internally defined threshold	0.790
Specificity at the internally defined threshold	0.921

At first, an AUC of 0.934 appears to show very strong transfer. It requires caution, however. The diagnosis could already be predicted from the image file's height and width alone with an AUC of 0.915. Image format and diagnosis were therefore linked in this dataset as well. The model might derive part of its performance from these technical properties rather than exclusively from changes in the lungs.

To reduce this image-size confounder, I first calculated a technical risk score from the image dimensions and divided the dataset into five groups with similar scores. Within each group, image size and diagnosis are less strongly linked. The AUCs calculated within those groups were weighted by the number of genuinely comparable positive-negative pairs. This produces a more conservative, size-adjusted AUC of 0.923. The procedure cannot remove the confounder completely, but it shows that the model's ranking is not explained by image dimensions alone.

The upper part of the figure shows the larger problem in this external evaluation: the probabilities themselves did not transfer. The model reported a mean pneumonia probability of 25.1 per cent, whereas the actual prevalence was 73.0 per cent. Even a predicted probability of approximately 25 per cent corresponded to pneumonia in around 93 per cent of cases. ECE consequently rose to 0.478.

A control calculation changed only the calibration intercept to reflect the external prevalence. ECE fell to 0.164 and the mean prediction increased to 70.8 per cent. This is not a post-hoc repair of the model. The calculation uses the already known prevalence of the complete external dataset and only shows how much of the miscalibration can be explained by the large prevalence difference. Before any real deployment, calibration would have to be estimated on a separate, representative target population and then evaluated independently.

The internally selected threshold was also retained unchanged. It achieved a sensitivity of 79.0 per cent and a specificity of 92.1 per cent. Nevertheless, 899 of the 2,357 images classified as negative contained pneumonia. The central observation is therefore that the model still ranks the paediatric images comparatively well by risk, but its numerical probabilities and the resulting yes-or-no decision do not transfer reliably to this population.

The dataset contains neither reliable acquisition-projection information nor bounding boxes. The projection confounder and localisation head therefore cannot be evaluated externally here. The data also include only children aged one to five years; these results make no claim about external performance in adults.

Adult Chest Radiographs: VinDr-CXR

The second external evaluation uses the VinDr-CXR dataset. It contains 18,000 frontal chest radiographs from adult patients at two hospitals in Vietnam. For this evaluation, I used the 15,000 images from the VinDr training split. Here, “training” refers only to the division chosen by the dataset authors: every image was new to my model and used exclusively for external evaluation. Three radiologists independently read each image, and local abnormalities were additionally marked with rectangles. The dataset's construction and annotation process are described in the original publication.

VinDr does not use exactly the same target as the RSNA dataset. As the closest available match, I combined the findings Lung Opacity, Consolidation, and Infiltration. Before the first model run, I specified that an image would count as positive as soon as at least one of the three radiologists marked one of these findings. This produced 1,588 positive images. In 750 of them—47 per cent—the mark came from only one radiologist.

Metric	At least 1 of 3 radiologists—specified in advance	At least 2 of 3—post hoc
Images	15,000	15,000
Positive images	1,588 (10.6%)	838 (5.6%)
Ensemble AUC	0.837 (95% CI: 0.825–0.848)	0.876 (95% CI: 0.863–0.888)
AUC after controlling for image geometry	0.794	0.840
Localisation-head point AUC	0.740 (95% CI: 0.727–0.752)	0.802 (95% CI: 0.788–0.816)
ECE of the unchanged calibration	0.024	0.060
Sensitivity at the internally defined threshold	0.591	0.714
Specificity at the internally defined threshold	0.892	0.873

The first column is the actual prespecified analysis. A minimum point AUC of 0.75 had been defined for the localisation head. This value comes from a fixed position template derived from the training annotations that never looks at the current radiograph. The head achieved 0.740 externally; even the upper confidence limit of 0.752 remained approximately level with this template. The primary criterion therefore failed. Internally, the same head had achieved 0.912.

After considering the prespecified binary target structure—pneumonia versus no pneumonia—I chose to add a more plausible definition: an image counts as positive only when at least two of the three radiologists identified the finding. A single dissenting annotation therefore no longer makes an image positive automatically. Under this majority rule, point AUC rose to 0.802. The decision was made only after the first result, however. It therefore remains a post-hoc analysis and does not replace the prespecified evaluation; both numbers belong in the report.

The detailed breakdown explains at least part of the difference. On images where only one radiologist had marked an opacity, localisation-head point AUC was 0.669. With agreement from two radiologists it rose to 0.754, and with unanimous agreement to 0.849. The more clearly the radiologists agreed on a finding, the better the localisation head identified its marked region. This indicates that the reference standard had a strong influence, but it does not prove that inconsistent labels alone caused the performance drop.

In VinDr, the target class could also be predicted from the original image geometry: depending on the label rule, the dimension-only score achieved an AUC between 0.768 and 0.782. Classification AUC was therefore again calculated within five groups of similar image geometry and weighted by the number of comparable positive-negative pairs. This left an AUC of 0.794 under the prespecified rule and 0.840 under the majority rule. Unlike the localisation head, pure risk ranking transferred comparatively well to the adult population.

Calibration behaved very differently from the paediatric dataset. Under the prespecified rule, ECE was 0.024, relatively close to the internal value of 0.009. A mathematical adjustment to the lower prevalence actually made it worse, increasing ECE to 0.050. Under the majority rule, ECE was initially 0.060 and fell to 0.026 after the same type of prevalence shift. Model probabilities therefore depend not only on the population being examined, but also on the level of radiological agreement used to define truth.

At the unchanged internal threshold, sensitivity was only 59.1 per cent and specificity was 89.2 per cent under the prespecified rule. Under the majority rule, the corresponding values were 71.4 and 87.3 per cent. The lower sensitivity is not simply a calibration failure: findings marked by only one radiologist—and therefore presumably subtler findings—often received low model scores. Threshold performance is consequently not a fixed property of a model; it also depends on the case spectrum and reference standard.

VinDr does not answer every open question either. The 512-pixel release used here contains no DICOM headers and therefore no reliable acquisition-projection field. The remaining projection confounder cannot be measured externally on this dataset. VinDr also contains frontal images only. The adult validation thus evaluates classification and localisation in a new population, but neither lateral images nor the external transfer of the projection signal.

Approaches That Were Rejected

Several variants were measured and subsequently rejected:

• A pixel-accurate lung mask removed 12.9 per cent of the annotated rectangle area and left the cardiac silhouette behind as a new technical shape.
• Per-image normalisation and CLAHE strengthened the projection cue.
• An adaptive lung crop re-encoded acquisition projection as a magnification factor. A fixed crop did not improve the target metric either.
• Stronger geometric augmentation removed only around 24 per cent of the intended size cue and did not alter the confounder reliably.
• A resolution of 512 rather than 224 pixels improved neither diagnosis nor confounding. The unwanted cue moved from global intensity to fine texture.
• Very strong brightness and contrast jitter improved robustness to brightness changes but did not reliably reduce the projection channel.

Raw predictions, evaluations, and reasons for rejection are stored for every discarded experiment in the experiment archive on GitHub. The Git history additionally documents when prespecifications, runs, and assessments were created.

Conclusion

This project showed me how closely model, data, and evaluation are connected. Technically, ResNet-18 could be adapted to chest radiographs. The second head learned a useful localisation map, and the ensemble achieved a stratified AUC of 0.8687 on the internal holdout. Without the other evaluations, however, that number would be easy to overestimate.

I learned the most from the points where plausible ideas did not work. The model used acquisition projection even though it was not the target finding. Image masking, per-image normalisation, and higher resolution did not solve the problem. Weighted sampling reduced the association, but also reduced diagnostic performance. Risk ranking survived in both external datasets: an image-size-adjusted AUC of 0.923 on Kermany and 0.794 on VinDr under the prespecified label rule. Calibration failed on the paediatric images, but held much better in the adult dataset.

The localisation head revealed another limitation. Under the prespecified VinDr analysis, it remained just below a template that never looks at the image. It exceeded that template under a majority rule introduced later, but the better result cannot retrospectively replace the original failure.

These are precisely the relationships I wanted to understand through this hobby project. The remaining confounder, rejected experiments, inconsistent reference labels, and partial transfer are therefore as much a part of the result as the ensemble AUC.

Glossary

Term	Explanation
AUC	Probability that a positive case is scored higher than a negative case; 0.5 represents chance.
Augmentation	Artificial but plausible alteration of a training image, such as a slight rotation or brightness change.
Batch	Small group of images processed together before the model weights are updated.
Backpropagation	Backward calculation used to determine how the model weights affected the loss.
Binary cross-entropy	Loss function for two classes that penalises confident incorrect predictions particularly strongly.
Calibration	Adjustment of the probability scale so that predictions and observed frequencies correspond more closely.
CNN / convolutional network	Neural network that processes images using small learned filters.
Confounder	Nuisance factor associated with the target class that provides the model with an unwanted shortcut.
ECE	Mean discrepancy between predicted probability and observed frequency; lower is better.
Ensemble	Several models solve the same task and their outputs are combined.
Epoch	One complete pass through all training examples.
Fine-tuning	Continued training of a pretrained model on a new task.
Fold	One partition of a dataset used in cross-validation.
Grad-CAM	Post-hoc map of the regions that influenced the classification score.
Holdout	Final evaluation set left untouched until development is complete.
Logit	Raw network output before conversion into a probability.
Loss	Number indicating how wrong the current model output is.
Parameter / weight	Learned number that determines how input signals are combined in the model.
Platt calibration	Monotonic logistic curve with two parameters that converts model scores into better-calibrated probabilities.
ResNet	Network architecture with shortcut connections that facilitate the training of deeper models.
Sigmoid	Function that maps any raw value to the interval from 0 to 1.
Transfer learning	Reuse of a starting model previously trained on a larger task.

Reproducibility and Code

• GitHub repository with project overview and results
• Training script for the deployed model
• Training code and two-head model
• Model implementation used by the web application
• Calibration, weights, and threshold
• Archive of rejected experiments
• Result of the one-time holdout run
• Experiment on the second head and Grad-CAM
• External Kermany evaluation
• Machine-readable report of the Kermany evaluation
• External VinDr evaluation
• Prespecification of the VinDr evaluation
• Result, majority rule, and correction of the calibration calculation
• Machine-readable reports of the VinDr evaluation

The architecture figure can be reproduced with generate_architecture_figure.py. The three-example comparison is generated by generate_example_figure.py. The external-validation plots are generated directly from the saved predictions by rsna_extern_kermany_bild.py and rsna_extern_vindr_bild.py.