How it performs

TideWatch against flexiclass.

The same held-out flood images, sent to both: TideWatch, trained for this question, and flexiclass, Accelerant’s general decision service, asked it zero-shot. Every metric, with its uncertainty.

Measured 2026-11-20. How this was measured ↓

Aerial · 3,034 test images

TideWatch0.994
flexiclass0.970

ROC-AUC · accuracy 98.1% against 96.3%

Satellite · 594 test images

TideWatch0.999
flexiclass0.894

ROC-AUC · accuracy 98.0% against 83.3%

Aerial

Aerial: 3,034 test images.

Held-out drone and helicopter photos, never used to train TideWatch; the same files sent to both.

False positive rateTrue positive rate011
ROC curve
RecallPrecision011
Precision-recall curve

TideWatch flexiclass

Every metric

MetricTideWatchflexiclassDifferencep
ROC-AUCRanking: chance a flooded image scores above a dry one0.9940.989–0.9970.9700.957–0.9810.0240.015–0.035<0.001
PR-AUCAverage precision across recall0.9710.958–0.9810.9020.875–0.9290.0690.048–0.091<0.001
AccuracyShare of images answered right0.9810.976–0.9850.9630.956–0.9700.0180.011–0.025<0.001
Balanced accuracyMean of recall and specificity0.9580.943–0.9710.9010.880–0.9240.0560.038–0.074<0.001
PrecisionOf the images called flooded, the share that are0.8880.855–0.9190.8140.768–0.8570.0740.027–0.1230.008
RecallOf the flooded images, the share found0.9280.898–0.9550.8240.781–0.8680.1040.069–0.139<0.001
SpecificityOf the dry images, the share called dry0.9870.983–0.9910.9790.973–0.9840.0080.002–0.0150.020
F1Precision and recall, equally weighted0.9080.884–0.9280.8190.784–0.8510.0890.056–0.119<0.001
F2Recall weighted 4 times precision: a missed flood costs more0.9200.895–0.9410.8220.786–0.8590.0980.065–0.128<0.001
MCCCorrelation of answer and truth, −1 to 10.8970.871–0.9200.7980.760–0.8340.0990.063–0.132<0.001
Brier scoreMean squared error of the probability; lower is better0.0150.013–0.0190.0300.025–0.035-0.015-0.019–-0.010<0.001
Calibration errorGap between stated probability and flood rate; lower is better0.0160.013–0.0210.0370.031–0.043-0.021-0.026–-0.015<0.001

95% intervals from 1,000 bootstrap resamples of whole groups (a tile’s before and after together); difference is TideWatch minus flexiclass, its p-value 2-sided. A shaded cell is the better model where the difference is significant at 5%. Decisions at a probability of 0.5.

McNemar’s exact test on the decisions: TideWatch alone right on 86 images, flexiclass alone on 32; p = <0.001. flexiclass’s own reliability, read as a probability, has a calibration error of 0.066.

TideWatch

Called floodedCalled dry
Flooded28522
Dry362,691

flexiclass

Called floodedCalled dry
Flooded25354
Dry582,669

Accuracy by subset

By dataset
LevelImagesFloodedTideWatchflexiclass
AIDER95878100.0%99.6%
The AIFloodSense dataset9595100.0%100.0%
FloodNet4755299.2%97.5%
LADI1,0498294.9%94.3%
RescueNet4570100.0%92.1%
By class
LevelImagesFloodedTideWatchflexiclass
collapsed building760100.0%100.0%
fire780100.0%98.7%
flooded22922990.4%76.9%
flooded areas7878100.0%98.7%
normal6540100.0%100.0%
not flooded1,847098.1%97.0%
traffic incident720100.0%97.2%

On data TideWatch never saw

Each left out of TideWatch’s training in turn
Left outImagesTideWatch accuracyflexiclass accuracyTideWatch ROC-AUCflexiclass ROC-AUC
The AIFloodSense dataset47098.7%96.4%——
AIDER6,43395.8%99.5%0.9991.000
FloodNet2,34396.8%96.6%0.9900.988
LADI9,96992.3%90.5%0.9280.898
RescueNet4,49499.8%91.0%——

All held-out events together

MetricTideWatchflexiclassDifferencep
ROC-AUCRanking: chance a flooded image scores above a dry one0.9470.942–0.9530.9380.933–0.9430.0090.005–0.014<0.001
PR-AUCAverage precision across recall0.8720.862–0.8840.8170.805–0.8290.0560.047–0.064<0.001
AccuracyShare of images answered right0.9530.950–0.9550.9370.934–0.9400.0150.012–0.018<0.001
Balanced accuracyMean of recall and specificity0.8820.874–0.8900.8380.831–0.8470.0430.037–0.050<0.001
PrecisionOf the images called flooded, the share that are0.8260.812–0.8400.7740.759–0.7900.0510.034–0.069<0.001
RecallOf the flooded images, the share found0.7870.773–0.8030.7060.692–0.7230.0810.068–0.094<0.001
SpecificityOf the dry images, the share called dry0.9760.974–0.9780.9700.968–0.9730.0060.003–0.009<0.001
F1Precision and recall, equally weighted0.8060.795–0.8170.7390.727–0.7510.0670.056–0.078<0.001
F2Recall weighted 4 times precision: a missed flood costs more0.7950.782–0.8080.7190.706–0.7340.0760.064–0.087<0.001
MCCCorrelation of answer and truth, −1 to 10.7790.767–0.7920.7040.691–0.7180.0750.062–0.088<0.001
Brier scoreMean squared error of the probability; lower is better0.0380.036–0.0400.0500.048–0.053-0.012-0.014–-0.010<0.001
Calibration errorGap between stated probability and flood rate; lower is better0.0250.022–0.0270.0270.025–0.030-0.002-0.006–0.0010.188

Augmentation, in training and at test time

TideWatchAccuracyROC-AUCF1BrierNever seen: accuracyNever seen: ROC-AUC
TideWatch, shipped0.9810.976–0.9850.9940.989–0.9970.9080.884–0.9280.0150.9530.947
Shipped + test-time augmentation0.9830.978–0.9870.9950.990–0.9970.9150.892–0.9350.0140.9620.949

Test-time augmentation averages the original with 3 label-safe views (flips, a small rotation or a crop), so it costs 4 encodes per image. Augmented training adds 3 transformed copies of every training image. Shaded: the best on the clean test split.

On data never seen, test-time augmentation against none, the same head and images: accuracy +0.9 points (95% interval +0.8 to +1.1, p = <0.001), ROC-AUC +0.002 (p = <0.001); McNemar: with it alone right on 280 images, without it on 61, p = <0.001. The service scores aerial images this way by default (TIDEWATCH_TTA_MODELS).

Robustness: the same images, transformed

50%60%70%80%90%100%Clean imagesDarker (60%)Brighter (140%)Lower contrastDesaturatedHue shift ⚠Colour jitterGrayscaleEqualizedAutocontrastPosterized (3 bits)Solarized ⚠Inverted ⚠Gaussian blurSharpenedGaussian noiseJPEG quality 20MirroredRotated 20°Cropped to halfShiftedShearedZoomed outPerspectiveElastic warpPatch erasedRandAugmentAugMix
Accuracy on each transformed copy of the same images

TideWatch flexiclass

TransformImagesTideWatchflexiclass
Clean images55496.9%AUC 0.997 · flipped 0.0%92.4%AUC 0.976 · flipped 0.0%
Darker (60%)55496.2%AUC 0.996 · flipped 1.1%91.5%AUC 0.981 · flipped 2.7%
Brighter (140%)55496.2%AUC 0.996 · flipped 1.8%90.6%AUC 0.972 · flipped 3.2%
Lower contrast55495.8%AUC 0.996 · flipped 1.4%91.7%AUC 0.979 · flipped 2.9%
Desaturated55495.5%AUC 0.996 · flipped 2.2%91.3%AUC 0.974 · flipped 2.2%
Hue shift ⚠55490.4%AUC 0.991 · flipped 6.9%87.7%AUC 0.962 · flipped 5.1%
Colour jitter55495.5%AUC 0.996 · flipped 1.4%91.9%AUC 0.977 · flipped 2.0%
Grayscale55490.8%AUC 0.991 · flipped 6.9%86.3%AUC 0.933 · flipped 7.9%
Equalized55493.5%AUC 0.994 · flipped 4.2%91.2%AUC 0.959 · flipped 4.2%
Autocontrast55497.1%AUC 0.997 · flipped 0.5%93.0%AUC 0.978 · flipped 1.3%
Posterized (3 bits)55493.5%AUC 0.994 · flipped 3.8%87.7%AUC 0.941 · flipped 7.2%
Solarized ⚠55493.7%AUC 0.993 · flipped 3.6%89.4%AUC 0.952 · flipped 6.0%
Inverted ⚠55483.6%AUC 0.977 · flipped 14.1%83.4%AUC 0.911 · flipped 12.6%
Gaussian blur55495.3%AUC 0.996 · flipped 2.0%91.5%AUC 0.968 · flipped 3.8%
Sharpened55496.2%AUC 0.997 · flipped 1.1%91.7%AUC 0.969 · flipped 2.2%
Gaussian noise55494.6%AUC 0.993 · flipped 2.7%90.6%AUC 0.969 · flipped 2.9%
JPEG quality 2055495.3%AUC 0.996 · flipped 2.0%91.7%AUC 0.971 · flipped 2.5%
Mirrored55497.3%AUC 0.997 · flipped 0.7%92.2%AUC 0.976 · flipped 2.3%
Rotated 20°55494.8%AUC 0.996 · flipped 2.9%91.2%AUC 0.969 · flipped 2.3%
Cropped to half55496.0%AUC 0.993 · flipped 2.7%90.6%AUC 0.965 · flipped 4.3%
Shifted55496.2%AUC 0.997 · flipped 2.5%91.9%AUC 0.977 · flipped 2.7%
Sheared55495.7%AUC 0.997 · flipped 1.6%91.3%AUC 0.975 · flipped 1.8%
Zoomed out55493.1%AUC 0.996 · flipped 4.2%91.2%AUC 0.970 · flipped 3.4%
Perspective55494.6%AUC 0.997 · flipped 2.7%91.0%AUC 0.970 · flipped 4.0%
Elastic warp55495.8%AUC 0.996 · flipped 1.4%92.6%AUC 0.969 · flipped 3.1%
Patch erased55493.3%AUC 0.997 · flipped 4.0%91.3%AUC 0.971 · flipped 3.2%
RandAugment55495.5%AUC 0.996 · flipped 2.9%90.3%AUC 0.967 · flipped 3.2%
AugMix55495.7%AUC 0.994 · flipped 1.6%91.7%AUC 0.972 · flipped 2.5%

150 flooded and 150 dry test images, each under 27 torchvision transforms at a fixed strength. Flipped: the share of answers that changed from the clean image’s. ⚠ marks a transform that can change what the picture shows: brown floodwater stops looking like water.

Satellite

Satellite: 594 test images.

Held-out satellite tiles, never used to train TideWatch; the same files sent to both.

False positive rateTrue positive rate011
ROC curve
RecallPrecision011
Precision-recall curve

TideWatch flexiclass

Every metric

MetricTideWatchflexiclassDifferencep
ROC-AUCRanking: chance a flooded image scores above a dry one0.9990.998–1.0000.8940.843–0.9380.1050.062–0.157<0.001
PR-AUCAverage precision across recall0.9990.997–1.0000.8980.859–0.9320.1010.067–0.140<0.001
AccuracyShare of images answered right0.9800.966–0.9920.8330.792–0.8670.1460.113–0.184<0.001
Balanced accuracyMean of recall and specificity0.9740.957–0.9900.7920.753–0.8280.1830.149–0.221<0.001
PrecisionOf the images called flooded, the share that are1.0001.000–1.0000.9790.951–1.0000.0210.000–0.0490.104
RecallOf the flooded images, the share found0.9490.915–0.9790.5910.515–0.6640.3570.288–0.433<0.001
SpecificityOf the dry images, the share called dry1.0001.000–1.0000.9920.980–1.0000.0080.000–0.0200.104
F1Precision and recall, equally weighted0.9740.955–0.9890.7370.675–0.7920.2360.184–0.298<0.001
F2Recall weighted 4 times precision: a missed flood costs more0.9590.930–0.9830.6420.569–0.7100.3160.253–0.390<0.001
MCCCorrelation of answer and truth, −1 to 10.9580.930–0.9820.6690.603–0.7290.2900.231–0.352<0.001
Brier scoreMean squared error of the probability; lower is better0.0170.008–0.0290.1320.103–0.166-0.114-0.148–-0.086<0.001
Calibration errorGap between stated probability and flood rate; lower is better0.0220.011–0.0360.1250.094–0.165-0.103-0.140–-0.074<0.001

95% intervals from 1,000 bootstrap resamples of whole groups (a tile’s before and after together); difference is TideWatch minus flexiclass, its p-value 2-sided. A shaded cell is the better model where the difference is significant at 5%. Decisions at a probability of 0.5.

McNemar’s exact test on the decisions: TideWatch alone right on 87 images, flexiclass alone on 0; p = <0.001. flexiclass’s own reliability, read as a probability, has a calibration error of 0.101.

TideWatch

Called floodedCalled dry
Flooded22312
Dry0359

flexiclass

Called floodedCalled dry
Flooded13996
Dry3356

Accuracy by subset

By dataset
LevelImagesFloodedTideWatchflexiclass
SpaceNet82022595.5%91.1%
xBD39221099.2%79.3%
By class
LevelImagesFloodedTideWatchflexiclass
post flood26520796.2%74.0%
post flood crops282892.9%3.6%
pre flood3010100.0%99.0%
By flood event
LevelImagesFloodedTideWatchflexiclass
germany-flooding541390.7%79.6%
Hurricane Florence, 20188245100.0%61.0%
Hurricane Harvey, 20171126497.3%78.6%
louisiana-flooding1481297.3%95.3%
Midwest floods, 20194625100.0%84.8%
Nepal floods, 201715276100.0%88.2%

On data TideWatch never saw

Each left out of TideWatch’s training in turn
Left outImagesTideWatch accuracyflexiclass accuracyTideWatch ROC-AUCflexiclass ROC-AUC
xBD2,63081.2%79.8%0.8750.901
Hurricane Florence, 201857478.6%54.9%0.9930.806
Hurricane Harvey, 201772695.0%87.1%0.9990.926
Midwest floods, 201930692.5%80.4%0.9810.884
Nepal floods, 20171,02496.3%88.5%0.9990.981
SpaceNet81,37770.4%90.0%0.9080.807
germany-flooding37186.3%81.4%0.9490.721
louisiana-flooding1,00678.7%93.1%0.9450.888

All held-out events together

MetricTideWatchflexiclassDifferencep
ROC-AUCRanking: chance a flooded image scores above a dry one0.8970.882–0.9110.8880.870–0.9060.009-0.002–0.0180.088
PR-AUCAverage precision across recall0.8690.855–0.8850.8920.878–0.906-0.022-0.035–-0.011<0.001
AccuracyShare of images answered right0.8270.814–0.8390.8330.819–0.848-0.006-0.020–0.0080.402
Balanced accuracyMean of recall and specificity0.8270.813–0.8410.7910.777–0.8080.0350.022–0.048<0.001
PrecisionOf the images called flooded, the share that are0.7580.740–0.7760.9740.963–0.984-0.216-0.236–-0.196<0.001
RecallOf the flooded images, the share found0.8250.799–0.8500.5930.565–0.6250.2320.210–0.254<0.001
SpecificityOf the dry images, the share called dry0.8280.816–0.8410.9900.985–0.994-0.162-0.174–-0.149<0.001
F1Precision and recall, equally weighted0.7900.773–0.8060.7370.715–0.7620.0530.031–0.073<0.001
F2Recall weighted 4 times precision: a missed flood costs more0.8110.790–0.8320.6430.617–0.6730.1680.145–0.188<0.001
MCCCorrelation of answer and truth, −1 to 10.6450.618–0.6710.6670.642–0.693-0.022-0.047–0.0040.092
Brier scoreMean squared error of the probability; lower is better0.1490.137–0.1610.1340.122–0.1450.0150.004–0.0260.006
Calibration errorGap between stated probability and flood rate; lower is better0.1370.126–0.1490.1230.109–0.1350.0150.001–0.0280.042

Augmentation, in training and at test time

TideWatchAccuracyROC-AUCF1BrierNever seen: accuracyNever seen: ROC-AUC
TideWatch, shipped0.9800.966–0.9920.9990.998–1.0000.9740.955–0.9890.0170.8270.897
Shipped + test-time augmentation0.9800.965–0.9920.9990.998–1.0000.9740.954–0.9900.0160.8350.918

Test-time augmentation averages the original with 3 label-safe views (flips, a small rotation or a crop), so it costs 4 encodes per image. Augmented training adds 3 transformed copies of every training image. Shaded: the best on the clean test split.

On data never seen, test-time augmentation against none, the same head and images: accuracy +0.8 points (95% interval +0.1 to +1.4, p = 0.016), ROC-AUC +0.021 (p = <0.001); McNemar: with it alone right on 700 images, without it on 576, p = <0.001. The service scores satellite images this way by default (TIDEWATCH_TTA_MODELS).

Robustness: the same images, transformed

50%60%70%80%90%100%Clean imagesDarker (60%)Brighter (140%)Lower contrastDesaturatedHue shift ⚠Colour jitterGrayscaleEqualizedAutocontrastPosterized (3 bits)Solarized ⚠Inverted ⚠Gaussian blurSharpenedGaussian noiseJPEG quality 20MirroredRotated 20°Cropped to halfShiftedShearedZoomed outPerspectiveElastic warpPatch erasedRandAugmentAugMix
Accuracy on each transformed copy of the same images

TideWatch flexiclass

TransformImagesTideWatchflexiclass
Clean images37597.1%AUC 0.999 · flipped 0.0%79.7%AUC 0.904 · flipped 0.0%
Darker (60%)37596.5%AUC 0.999 · flipped 1.1%75.5%AUC 0.875 · flipped 6.9%
Brighter (140%)37597.3%AUC 0.998 · flipped 1.3%80.8%AUC 0.902 · flipped 5.3%
Lower contrast37596.0%AUC 0.999 · flipped 1.6%77.6%AUC 0.896 · flipped 8.0%
Desaturated37595.5%AUC 0.998 · flipped 2.1%76.3%AUC 0.892 · flipped 7.7%
Hue shift ⚠37594.9%AUC 0.986 · flipped 2.7%72.3%AUC 0.877 · flipped 14.9%
Colour jitter37596.5%AUC 0.998 · flipped 1.1%79.7%AUC 0.897 · flipped 4.3%
Grayscale37593.9%AUC 0.989 · flipped 3.7%67.7%AUC 0.845 · flipped 15.2%
Equalized37596.8%AUC 0.994 · flipped 3.5%71.5%AUC 0.837 · flipped 18.4%
Autocontrast37597.1%AUC 0.999 · flipped 1.6%80.0%AUC 0.908 · flipped 5.1%
Posterized (3 bits)37595.7%AUC 0.995 · flipped 4.5%66.7%AUC 0.753 · flipped 18.4%
Solarized ⚠37597.1%AUC 0.994 · flipped 3.2%79.2%AUC 0.889 · flipped 5.3%
Inverted ⚠37584.5%AUC 0.920 · flipped 14.1%58.1%AUC 0.713 · flipped 26.9%
Gaussian blur37595.2%AUC 0.993 · flipped 5.6%77.9%AUC 0.889 · flipped 8.8%
Sharpened37597.1%AUC 0.999 · flipped 0.5%80.0%AUC 0.903 · flipped 6.1%
Gaussian noise37595.2%AUC 0.997 · flipped 1.9%76.0%AUC 0.881 · flipped 8.0%
JPEG quality 2037593.9%AUC 0.997 · flipped 3.7%76.5%AUC 0.891 · flipped 9.1%
Mirrored37596.8%AUC 0.999 · flipped 0.8%79.5%AUC 0.902 · flipped 7.7%
Rotated 20°37596.0%AUC 0.997 · flipped 2.1%78.4%AUC 0.902 · flipped 9.3%
Cropped to half37595.5%AUC 0.983 · flipped 4.8%76.3%AUC 0.882 · flipped 9.3%
Shifted37597.1%AUC 0.997 · flipped 1.6%77.9%AUC 0.896 · flipped 6.7%
Sheared37595.7%AUC 0.998 · flipped 1.9%78.1%AUC 0.885 · flipped 6.4%
Zoomed out37596.5%AUC 0.998 · flipped 2.7%76.3%AUC 0.880 · flipped 7.7%
Perspective37596.0%AUC 0.998 · flipped 2.1%78.1%AUC 0.899 · flipped 8.0%
Elastic warp37597.1%AUC 0.997 · flipped 1.6%78.4%AUC 0.906 · flipped 6.1%
Patch erased37595.7%AUC 0.998 · flipped 1.9%75.5%AUC 0.891 · flipped 6.4%
RandAugment37595.5%AUC 0.994 · flipped 3.7%76.5%AUC 0.888 · flipped 9.1%
AugMix37596.5%AUC 0.997 · flipped 2.7%75.7%AUC 0.872 · flipped 9.3%

150 flooded and 150 dry test images, each under 27 torchvision transforms at a fixed strength. Flipped: the share of answers that changed from the clean image’s. ⚠ marks a transform that can change what the picture shows: brown floodwater stops looking like water.

Method

How this was measured.

flexiclass is tuned for many questions, not this one; a trained model is expected to lead on its own task. The test deployment of flexiclass records submitted images for its evaluation.