Pythia: a framework for the automated analysis of web hosting environments
Files
Accepted manuscript
Date
2019-05-13
DOI
Authors
Matic, Srdjan
Tyson, Gareth
Stringhini, Gianluca
Version
OA Version
Accepted manuscript
Citation
Srdjan Matic, Gareth Tyson, Gianluca Stringhini. 2019. "Pythia: a Framework for the Automated Analysis of Web Hosting Environments." The Web Conference (WWW)
Abstract
A common approach when setting up a website is to utilize
third party Web hosting and content delivery networks. Without
taking this trend into account, any measurement study
inspecting the deployment and operation of websites can be
heavily skewed. Unfortunately, the research community lacks
generalizable tools that can be used to identify how and
where a given website is hosted. Instead, a number of ad hoc
techniques have emerged, e.g., using Autonomous System
databases, domain prefixes for CNAME records. In this work
we propose Pythia, a novel lightweight approach for identifying
Web content hosted on third-party infrastructures,
including both traditional Web hosts and content delivery
networks. Our framework identifies the organization to which
a given Web page belongs, and it detects which Web servers
are self-hosted and which ones leverage third-party services
to provide contents. To test our framework we run it on
40,000 URLs and evaluate its accuracy, both by comparing
the results with similar services and with a manually validated
groundtruth. Our tool achieves an accuracy of 90% and
detects that under 11% of popular domains are self-hosted.
We publicly release our tool to allow other researchers to
reproduce our findings, and to apply it to their own studies.
Description
License
Attribution 4.0 International