Classification Pipeline

SWIFT uses a two-stage classification pipeline with an additional fog counter-prompt system.

2
Pipeline stages
7
Weather categories
ViT-B/32
CLIP backbone
τ = 10
Temperature scaling
Stage 1

Indoor / Outdoor Filter

Filters out indoor images using 5 outdoor vs. 5 indoor prompts. Max cosine similarity per category determines the classification.

Stage 2

Weather Classification

Classifies outdoor images into 6 weather categories using 5 prompts each. Softmax with temperature scaling produces calibrated probabilities.

Fog Check

Counter-Prompt Margin

Separates true fog from confounders (clouds, blur, clear) using margin scoring: fog_score − max(confounder_scores) ≥ 0.15.

Stage 1: Indoor / Outdoor Classification

Images scoring higher on outdoor prompts proceed to weather classification. Indoor images are filtered out.

Outdoor 5 prompts
  • "an outdoor scene with trees and sky"
  • "a street view with buildings and roads"
  • "an open landscape with horizon visible"
  • "an exterior view of buildings and nature"
  • "an outdoor environment with natural lighting"
Indoor 5 prompts
  • "an indoor room with walls and ceiling"
  • "an interior space with artificial lighting"
  • "inside a building with closed environment"
  • "a room interior with furniture and walls"
  • "an enclosed indoor space"

Stage 2: Weather Classification

Outdoor images are classified into 6 standard weather categories. Each category uses 5 descriptive CLIP prompts.

☀️ Clear Day 5 prompts
  • "bright sunny day with blue sky and strong shadows"
  • "clear weather with bright sunlight and vivid colors"
  • "sunny outdoor scene with sharp contrast and blue sky"
  • "daylight scene with bright illumination and clear visibility"
  • "clear day with strong natural lighting and shadows"
☁️ Cloudy 5 prompts
  • "overcast sky with gray clouds covering the sun"
  • "cloudy weather with diffused lighting and gray sky"
  • "cloudy day with muted colors and soft shadows"
  • "overcast conditions with cloud cover and dim lighting"
  • "gray cloudy sky with uniform diffused light"
🌧️ Rainy 5 prompts
  • "wet streets with rain puddles and reflections"
  • "rainy weather with water drops and wet surfaces"
  • "rain scene with umbrellas and wet pavement"
  • "stormy weather with rain and dark clouds"
  • "wet conditions with rain and water on surfaces"
❄️ Snowy 5 prompts
  • "snow covered ground with white precipitation"
  • "winter scene with snow falling and white landscape"
  • "snowy weather with snowflakes and winter conditions"
  • "white snow on trees and ground in winter"
  • "cold snowy scene with snow accumulation"
🌙 Night 5 prompts
  • "nighttime scene with artificial street lighting"
  • "dark evening with illuminated buildings and street lights"
  • "night photography with artificial lighting and darkness"
  • "after dark scene with electric lights and dark sky"
  • "nighttime urban scene with bright lights against dark background"
🌅 Dawn / Dusk 5 prompts
  • "golden hour lighting with warm orange and pink sky"
  • "sunrise or sunset with dramatic sky colors"
  • "twilight scene with gradient sky from light to dark"
  • "dawn or dusk with low sun angle and warm lighting"
  • "golden hour photography with soft warm light"

Fog Counter-Prompt System

Fog is distinguished from visual confounders using a margin-based scoring system. An image is classified as foggy when: fog_score − max(cloudy, blurred, clear) ≥ 0.15

🌫️ Fog (Target) 3 prompts
  • "a photo of a dense ground-level fog obscuring distant objects in an outdoor scene"
  • "a photo of heavy haze over a landscape with very low visibility"
  • "a photo of thick fog covering the ground in a countryside scene"
Mean embedding of all prompts → cosine similarity = fog_score
☁️ Cloudy (Confounder) 2 prompts
  • "a photo of an overcast cloudy sky with no fog near the ground"
  • "a photo of thick clouds in the sky above a clear landscape"
📷 Blurred (Confounder) 3 prompts
  • "a photo of a blurred portrait of a person"
  • "a photo of a defocused close-up of a face"
  • "a photo of a blurred object"
🔆 Clear (Confounder) 3 prompts
  • "a photo of a clear day with high visibility"
  • "a photo of a crisp landscape with no haze or fog"
  • "a photo of a clear blue sky"

Severe Weather Classification

Alternative prompt set for detecting extreme weather conditions. Used when fine-grained severity distinction is needed.

⛈️ Severe Rain 5 prompts
  • "heavy torrential rain with water pouring down intensely"
  • "severe rainstorm with visible raindrops falling rapidly"
  • "extreme rainfall with rain drops covering the camera lens"
  • "monsoon-like conditions with heavy downpour and flooding"
  • "intense rain with water streaming down surfaces and puddles forming quickly"
🌨️ Severe Snow 5 prompts
  • "heavy snowfall with thick snow accumulating on the ground"
  • "blizzard conditions with snow falling heavily and wind blowing"
  • "severe snowstorm with snow piling up rapidly on surfaces"
  • "extreme winter weather with dense snow falling continuously"
  • "intense snow with whiteout conditions and deep snow drifts"
🌫️ Severe Fog 5 prompts
  • "dense fog with visibility less than 5 meters obscuring everything"
  • "thick fog with zero visibility and objects barely visible at close range"
  • "severe haze with extremely low visibility under 10 meters"
  • "impenetrable fog covering the landscape with minimal sight distance"
  • "heavy mist and fog with visibility reduced to just a few meters"

Image Generation Prompts

Text prompts and negative prompts used by diffusion-based augmentation strategies to generate weather-adverse training images. Three detail levels are used depending on the generation model.

🌫️ Foggy
Prompt

"Transform the scene to dense fog with volumetric haze, reducing visibility to 200-400 meters. Keep all traffic signs, road markings, buildings, and vehicles clearly visible with cool blue-gray lighting and soft, diffused shadows."

Negative

"Clear sky, harsh shadows, oversaturation, blurred signs, missing markings, cartoon style, daytime lighting."

❄️ Snowy
Prompt

"Make the scene snowy with snow covering roads, vehicles, and buildings. Preserve traffic sign legibility with slight snow dusting on edges and maintain road lane geometry under overcast gray sky and soft lighting."

Negative

"Visible green foliage, dry surfaces, sharp sunlight, vibrant colors, buried signs, unrealistic snow placement, summer atmosphere."

🌧️ Rainy
Prompt

"Apply heavy rain with wet asphalt reflections and visible rain streaks. Keep all traffic signs and road markings sharp and readable with subtle water droplets and low ambient light under overcast clouds."

Negative

"Dry ground, clear sky, no reflections, fog, snow, blurred signs, cartoon rain, unrealistic puddles."

🌅 Dawn / Dusk
Prompt

"Change lighting to dawn or dusk with long soft shadows and colored sky gradients from orange to purple. Preserve illuminated traffic signs and building outlines with warm or cool low-angle sunlight."

Negative

"Midday sun, harsh shadows, overcast, night darkness, neon colors, unreadable signs."

🌙 Night
Prompt

"Convert to night with artificial warm street lighting and cool headlights. Ensure traffic signs glow true to color and road markings remain visible under high contrast with deep shadows."

Negative

"Daylight, uniform brightness, dark or missing signs, overexposed lights, cartoon style."

☁️ Cloudy
Prompt

"Transform to overcast conditions with dense cloud cover and soft diffused light. Keep traffic sign colors vibrant and visible, preserve road markings and building textures with a subtle blue-gray color cast."

Negative

"Sunny sky, sharp shadows, vibrant colors, fog, rain, snow, faded signs, cartoon style."

Technical Parameters

Default configuration values for the SWIFT classification pipeline.

Parameter Value Description
CLIP ModelViT-B/32Vision Transformer backbone for text-image similarity
Temperature (τ)10Softmax temperature scaling for probability calibration
Confidence Threshold0.3Minimum classification confidence to accept a prediction
Margin Threshold0.1Minimum margin between top-1 and top-2 categories
Fog Margin Threshold0.15Minimum fog vs. confounder margin for fog classification
Indoor/Outdoor Scoringmax cosine simMaximum cosine similarity across all prompts per category
Weather Scoringmax cosine simMaximum cosine similarity across per-category prompts
Fog Counter-Prompt Scoringmean embeddingMean embedding per class, then cosine similarity

View SWIFT Source Code on GitHub →