Skip to content
This repository was archived by the owner on Jun 24, 2025. It is now read-only.
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 14 additions & 15 deletions _posts/2023-02-07-varnish.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ Varnish Cache is a web application accelerator also known as a caching HTTP reve

## Varnish vs Squid

Squid and Varnish are both open source tools. Varnish with 908 GitHub stars and 216 forks on GitHub appears to be more popular than Squid with 560 GitHub stars and 202 GitHub forks. We use open source version of Varnish that lacks persistent storage and range requests. For most usecases this is completely sufficient as even caching in only 2GB of RAM still gives >99.5% cache hit rate.

Squid and Varnish are both open source tools. Varnish with 908 GitHub stars and 216 forks on GitHub appears to be more popular than Squid with 560 GitHub stars and 202 GitHub forks. Follow [squid-vs-varnish stackups](https://stackshare.io/stackups/squid-vs-varnish) for more details. We use open source version of Varnish that lacks persistent storage and range requests. For most usecases this is completely sufficient as even caching in only 2GB of RAM still gives >99.5% cache hit rate.
![varnish vs squid in popularity](/img/posts/popularity.svg)

<table>
Expand All @@ -42,23 +41,11 @@ High-performance HTTP accelerator. Varnish Cache is a web application accelerato

## Deployment Status

Currently, Varnish is deployed at AGLT2 and MWT2.
Currently, Varnish is deployed at two tier2 centers of the ATLAS experiment at the LHC: [AGLT2](https://www.aglt2.org/) and [MWT2](https://hep.uchicago.edu/atlas/mwt2/).

- **AGLT2:** Two instances were deployed as primary frontier caches in summer 2022. Since then there was no issues observed with the instances.
- **MWT2:** Recently deployed both a v4a and a v4cvmfs instances, serving all three sub-sites (UC, IU, UIUC) and Analysis Facility.

## Monitoring

Varnish is monitored in both Elasticsearch and [ATLAS MRTG monitoring](http://wlcg-squid-monitor.cern.ch/snmpstats/mrtgatlas2/indexatlas2.html) like squid does. Reports request/fetch, in/out data rate, CPU usage, objects and file descriptors.

### Varnish and Squid instance SNMP monitoring

![Varnish and Squid metrics](/img/posts/VarnishMetrics.svg)

### Varnish and Squid monitoring in Elasticsearch

![Varnish and squid in ES](/img/posts/squidVarnishInES.svg)

## Performance Test Results

### CVMFS
Expand Down Expand Up @@ -148,5 +135,17 @@ Shortest transaction: 0.00
</tr>
</table>

## Monitoring

Varnish is monitored in both [Elasticsearch](https://atlas-kibana.mwt2.org:5601/s/squids/app/dashboards?auth_provider_hint=anonymous1#/view/98e4a510-32c9-11ed-afcf-d91dad577662?_g=(filters%3A!()%2CrefreshInterval%3A(pause%3A!t%2Cvalue%3A0)%2Ctime%3A(from%3Anow-24h%2Fh%2Cto%3Anow))) and [ATLAS MRTG monitoring](http://wlcg-squid-monitor.cern.ch/snmpstats/mrtgatlas2/indexatlas2.html) like squid does. Reports request/fetch, in/out data rate, CPU usage, objects and file descriptors.

### Varnish and Squid instance SNMP monitoring

![Varnish and Squid metrics](/img/posts/VarnishMetrics.svg)

### Varnish and Squid monitoring in Elasticsearch

![Varnish and squid in ES](/img/posts/squidVarnishInES.svg)

## Summary
We showed in this post that Varnish is faster, needs less resources, and it is easy to monitor. It's a step forward toward efficiently caching Physics data over HTTP. Please refer to the chart repo on how to make deployments, and we encourage you to try this out and let us know if you have any feedback or suggestions that would help us improve this chart and make it more beneficial to users. For discussion, news and troubleshooting, the [SLATE Slack workspace](https://slack.slateci.io/) is the best place to reach us!