345 lines
10 KiB
Markdown
345 lines
10 KiB
Markdown
<div align="center">
|
|
<h1>Zephyr 🌲</h1>
|
|
|
|
<h6><i>real-time weather forecast service</i></h6>
|
|
|
|
[](https://github.com/ceticamarco/zephyr/actions/workflows/docker.yml)
|
|
[](https://github.com/ceticamarco/zephyr/actions/workflows/tests.yml)
|
|
|
|
</div>
|
|
|
|
Zephyr is a lightweight, real-time HTTP service that provides a simple way to gather
|
|
weather data and apply statistical analysis to past meteorological records. It is
|
|
written in Go using `net/http` and [OpenWeatherMap](https://openweathermap.org/)
|
|
for weather data.
|
|
|
|
I've built this service out of frustration with existing weather platforms
|
|
cluttered with ads, paywalls, clickbait content and other unnecessary distractions.
|
|
Zephyr is designed to be simple, fast and efficient, providing only the
|
|
weather data of a given location, without any additional nonsense.
|
|
|
|
This service communicates through a JSON API, making it suitable for
|
|
any kind of internet-based project or device. I already use it on a widget
|
|
on my phone, on my terminal, on the tmux's status bar, in a couple of
|
|
smart bedside clocks I've built and as a standalone web app.
|
|
|
|
## Basic Usage
|
|
As state before, Zephyr talks via HTTP using the JSON format. Therefore, you
|
|
can query it using any HTTP client of your choice. Below you can find some examples
|
|
using `curl`:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/weather/milan' | jq
|
|
```
|
|
|
|
which yield the following:
|
|
|
|
```json
|
|
{
|
|
"date": "Thursday, 2025/06/19",
|
|
"temperature": "33°C",
|
|
"condition": "Clear",
|
|
"feelsLike": "36°C",
|
|
"emoji": "☀️"
|
|
}
|
|
```
|
|
|
|
To get the results in imperial units, you can append the `i` query parameter to the
|
|
URL:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/weather/milan?i' | jq
|
|
```
|
|
|
|
which yields:
|
|
|
|
```json
|
|
{
|
|
"date": "Thursday, 2025/06/19",
|
|
"temperature": "65°F",
|
|
"condition": "Clear",
|
|
"feelsLike": "68°F",
|
|
"emoji": "☀️"
|
|
}
|
|
```
|
|
|
|
## Metrics
|
|
The `/metrics/:city` endpoint provides environmental metrics for a given city:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/metrics/taipei' | jq
|
|
```
|
|
|
|
which yields:
|
|
|
|
```json
|
|
{
|
|
"humidity": "23%",
|
|
"pressure": "1015 hPa",
|
|
"dewPoint": "6°C",
|
|
"uvIndex": "4",
|
|
"visibility": "10km"
|
|
}
|
|
```
|
|
|
|
As in the previous example, you can append the `i` query parameter to get results
|
|
in imperial units.
|
|
|
|
## Wind
|
|
The `/wind/:city` endpoint provides wind related information(such as speed and direction) for a given city:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/wind/bolzano' | jq
|
|
```
|
|
|
|
which yields:
|
|
|
|
```json
|
|
{
|
|
"arrow": "⬆️",
|
|
"direction": "S",
|
|
"speed": "13.0 km/h"
|
|
}
|
|
```
|
|
As in the previous examples, you can append the `i` query parameter to get results
|
|
in imperial units.
|
|
|
|
## Forecast
|
|
The `/forecast/:city` endpoint allows you to get the weather forecast of the
|
|
next 4 days. For example:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/forecast/Yakutsk' | jq
|
|
```
|
|
|
|
which yields:
|
|
|
|
```json
|
|
{
|
|
"forecast": [
|
|
{
|
|
"condEmoji": "🌧️",
|
|
"condition": "Rain",
|
|
"date": "Tuesday, 2025/05/06",
|
|
"feelsLike": "0°C",
|
|
"tempMax": "6°C",
|
|
"tempMin": "-2°C",
|
|
"windArrow": "↗️",
|
|
"windDirection": "SSW",
|
|
"windSpeed": "14.7 km/h"
|
|
},
|
|
{
|
|
"condEmoji": "☃️",
|
|
"condition": "Snow",
|
|
"date": "Wednesday, 2025/05/07",
|
|
"feelsLike": "7°C",
|
|
"tempMax": "9°C",
|
|
"tempMin": "2°C",
|
|
"windArrow": "↘️",
|
|
"windDirection": "NNW",
|
|
"windSpeed": "13.9 km/h"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
As in the previous examples, you can append the `i` query parameter to get results
|
|
in imperial units.
|
|
|
|
## Moon
|
|
|
|
The `/moon` endpoint provides the current moon phase and its emoji representation:
|
|
|
|
```sh
|
|
curl -s 'http://127.0.0.1:3000/moon' | jq
|
|
```
|
|
|
|
will yield
|
|
|
|
```json
|
|
{
|
|
"icon": "🌘",
|
|
"phase": "Waning Crescent",
|
|
"percentage": "44%"
|
|
}
|
|
```
|
|
|
|
> [!NOTE]
|
|
> To convert OpenWeatherMap's moon phase value to the illumination percentage,
|
|
> I've used the following formula:
|
|
>
|
|
> $$
|
|
> \sin(\pi \theta)^2 \times 100
|
|
> $$
|
|
|
|
## Statistical analysis
|
|
In addition to the weather data, Zephyr also provides statistical analysis of pase
|
|
meteorological records. This is done through the `/stats/:city` endpoint, which
|
|
returns additional information about the weather of the previous days such as
|
|
the average temperature, the maximum and minimum temperatures, the standard deviation,
|
|
the median and the mode.
|
|
|
|
This endpoint becomes available only after the service has collected enough
|
|
**updated** data for a given city. In particular, the services will require
|
|
**at least** two weather records **within the last 48 hours**. If these two
|
|
conditions aren't met, the service will refuse to provide statistical data.
|
|
|
|
After enough data has been collected in the in-memory database, you will be
|
|
able to query the statistics endpoint like this:
|
|
|
|
```sh
|
|
$ curl -s 'http://127.0.0.1:3000/stats/berlin' | jq
|
|
```
|
|
|
|
which yields:
|
|
|
|
```json
|
|
{
|
|
"min": "25°C",
|
|
"max": "25°C",
|
|
"count": 30,
|
|
"mean": "25°C",
|
|
"stdDev": "0.1821°C",
|
|
"median": "25°C",
|
|
"mode": "25°C",
|
|
"anomaly": null
|
|
}
|
|
```
|
|
After enough data has been collected, the service will also be able to detect
|
|
anomalies in the temperature data using a built-in statistical model.
|
|
|
|
For instance, two temperature spikes, such as `+34°C` and `-15°C`, with a mean of `25°C` and a standard deviation of `0.2°C`,
|
|
will be flagged as outliers by the model and will be reported as such:
|
|
|
|
```json
|
|
{
|
|
"min": "-15°C",
|
|
"max": "34°C",
|
|
"count": 32,
|
|
"mean": "24°C",
|
|
"stdDev": "7.1864°C",
|
|
"median": "25°C",
|
|
"mode": "25°C",
|
|
"anomaly": [
|
|
{
|
|
"date": "Sunday, 2025/06/01",
|
|
"temperature": "-15°C"
|
|
},
|
|
{
|
|
"date": "Wednesday, 2025/05/28",
|
|
"temperature": "34°C"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Anomaly Detection
|
|
The anomaly detection algorithm is based on a modified version of the
|
|
[Z-Score](https://en.wikipedia.org/wiki/Standard_score) algorithm, which uses the
|
|
[Median Absolute Deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation) to measure the variability
|
|
in a given sample of quantitative data. The algorithm can be summarized as follows(let $x$ be the sample):
|
|
|
|
|
|
$$
|
|
\tilde{x} = \text{median}({X})
|
|
$$
|
|
|
|
Compute The median absolute deviation
|
|
|
|
$$
|
|
\text{MAD} = \text{median}\{ |x_i - \tilde{x}| : \forall i = 0, \dots, n-1 \}
|
|
$$
|
|
|
|
Compute the (modified)Z-score
|
|
|
|
$$
|
|
z_i = \frac{0.6745 (x_i - \tilde{x})}{\text{MAD}}
|
|
\quad \forall i = 0, \dots, n-1
|
|
$$
|
|
|
|
Flag $x_i$ as an outlier if:
|
|
$$
|
|
|z_i| > 4.5
|
|
$$
|
|
|
|
and
|
|
|
|
$$
|
|
|x_i-\tilde{x}| \geq 8
|
|
$$
|
|
|
|
Here, $\Phi^{-1}(3/4) = \Phi^{-1}(0.75) \approx 0.6745$ reflects the fact
|
|
that 75% of values lie within $\approx 0.6745$ standard deviation, 4.5 represent a fixed threshold value and 8 represent the minimum absolute deviation that a value
|
|
must have from the median to be considered an outlier.
|
|
|
|
These constants have been fine-tuned to work well with the weather data of
|
|
a wide range of climates and to ignore daily temperature fluctuations while
|
|
still being able to detect significant anomalies.
|
|
|
|
Daily temperatures collected over a short time window(1/2 months, but not less than a few days)
|
|
// *should* be normally distributed. This algorithm only work under this assumption.
|
|
|
|
> [!IMPORTANT]
|
|
> The anomaly detection algorithm works under the assumption that the weather data
|
|
> is normally distributed(at least roughly), this might not be the case on datasets
|
|
> with a very small number of samples(e.g. few days of data) or with a large
|
|
> number of samples(e.g. multi-seasonal data).
|
|
|
|
My statistical benchmarks(QQ plots) show that the algorithm works
|
|
quite well when these conditions are met, and even with real world data,
|
|
the results were quite satisfactory. However, if it
|
|
start to produce false positives, you will need to dump the whole in-memory
|
|
database and start from scratch. I recommend to do this at every change of season.
|
|
|
|
## Embedded Cache System
|
|
To minimize the amount of requests sent to the OpenWeatherMap API, Zephyr provides an built-in,
|
|
in-memory cache data structure that stores fetched weather data. Each time a client requests
|
|
weather data for a given location, the service will first check if it's already available on the cache.
|
|
If it is found, the cached value will be returned, otherwise a new request will be sent to the OpenWeatherMap API
|
|
and the response will be returned to the client and stored in the cache for future use. Each cache entry
|
|
is valid for a fixed amount of time, which can be configured by setting the `ZEPHYR_CACHE_TTL` environment variable. Once
|
|
a cached entry expires, Zephyr will retrieve a new value from the OpenWeatherMap API and update the cache accordingly.
|
|
|
|
The cache system significantly improves the performance of the service by decreasing its latency. Additionally, it
|
|
also helps to reduce the number of API calls made to the OpenWeatherMap servers, which is quite important
|
|
if you are using their free tier.
|
|
|
|
## Configuration
|
|
Zephyr requires the following environment variables to be set:
|
|
|
|
| Variable | Meaning |
|
|
|----------------------|----------------------------------------|
|
|
| `ZEPHYR_PORT` | Listen port |
|
|
| `ZEPHYR_TOKEN` | OpenWeatherMap API key |
|
|
| `ZEPHYR_CACHE_TTL` | Cache time-to-live(expressed in hours) |
|
|
|
|
Each value must be set _before_ launching the application. If you plan to deploy Zephyr using
|
|
Docker, you can specify these variables in the `compose.yml` file.
|
|
|
|
You will also need an OpenWeatherMap API key, you can get one for free by following
|
|
the instructions [listed on their website](https://openweathermap.org/api).
|
|
|
|
> [!NOTE]
|
|
> Zephyr is designed to work with OpenWeatherMap's free tier. As long as you
|
|
> stay within the daily limits of 1,000 requests, you won't need to pay.
|
|
|
|
## Deploy
|
|
Zephyr can be deployed using Docker by just issuing the following command:
|
|
|
|
```sh
|
|
docker compose up -d
|
|
```
|
|
|
|
This will build the container image and start the service in detached mode. By default,
|
|
the service will be available at `http://127.0.0.1:3000`, but you can easily change this property
|
|
but editing the `compose.yml` as stated above.
|
|
|
|
## Unit tests
|
|
You can run the unit tests by issuing the following command:
|
|
|
|
```sh
|
|
go test ./... -v
|
|
```
|
|
|
|
## License
|
|
This software is released under the GPLv3 license. You can find a copy of the license with this repository or by visiting the [following page](https://choosealicense.com/licenses/gpl-3.0/). |