Tech-Powered OSINT: hunting for intelligence in open sources with Python

Chiin-Rui Tan, Dare Imam-Lawal

Algorithms Data Machine-Learning Open-Source Web Crawling

What is OSINT? And who cares?

• OSINT (Open-Source Intelligence) is the practice of gaining insight from publicly available sources to empower its end-users.

• Originating as a statecraft tool, broadening use of OSINT has demonstrated it’s potential for social impact, as evidenced by the work of citizen and research organisations like Bellingcat and The Human Rights Centre at UC Berkeley in exposing crimes against humanity.

Current challenges & limitations

• Lack of Modernisation: OSINT’s lagging adaption to contemporary open source landscapes has created persistent operational deficiencies, as made apparent by Western Intelligence agencies’ failure to predict the social-media mobilised Arab Spring uprisings.

• Disinformation & Volumes of Data: public digital platforms are increasingly an information battleground between facts and alternative facts, as Industry 3.0’s advancing democratisation of technology facilitates User-Generated Content by good and bad actors alike. And this noisy Big Data challenge is set to scale dramatically with additional AI-Generated Content and IoT sensor data in the 5G Industry 4.0 era.

Opportunities for improvement

• In these complex conditions both possibilities and hurdles have never been higher. This talk focuses on getting started tackling the technical hurdle.

• We will present realistic OSINT scenarios, outline the advantages of adopting techno-centric approaches, and walkthrough practical Python-enabled workflows, accessible to those with basic proficiency and no prior domain knowledge.


• The aim of sharing this topic is for public interest, the real-world Python use-case, and to incentivise the community to get into ethical OSINT hacking!

Type: Talk (30 mins); Python level: Beginner; Domain level: Beginner

Chiin-Rui Tan

Rho Zeta AI

Winner of UK Entrepreneur 2018 at the Data Leaders Awards, Chiin is a technologist with 12 years’ experience pioneering digital and data innovation, skunk works, and building tech teams, cumulatively delivering multi-million pound value. Originally an Economics graduate from Cambridge University, Chiin taught herself to code later in her twenties. She went on to become the first Data Scientist to work for the UK Foreign & Commonwealth Office taking the role of Head of Data Science, under Cyber & National Security. Chiin used Python and R to deliver the organisation’s first AI project, new OSINT capabilities and initiatives, gaining MP and senior adoption before leaving to set-up her own startup.

Now an EdTech founder, Chiin is working with schools not spies (!), testing products she and her team at Rho Zeta AI have developed to democratise hi-tech education to youth, for the goal of enabling future universal AI literacy.

An active tech community leader, Chiin founded the AI Club for Gender Minorities, a 700-member meetup of mainly Machine Learning, Data Science, and Data Engineering Pythonistas, where she also successfully mentors many individuals without formal STEM experience into their first tech job. She was previously jointly awarded a Linux Foundation grant for R user group expansion.

Dare Imam-Lawal

Rho Zeta AI

Dare is a final year Aerospace Engineering PhD student at Queen Mary, University of London. His research focuses on developing and applying novel Machine Learning techniques to solve aeronautical optimisation problems.
A self-taught programmer, Dare has been coding for several years, winning awards from the Royal Aeronautical Society (RAeS) and Institute of Mechanical Engineers (IMechE) for his contribution to the development of an open source Computational Fluid Dynamics (CFD) solver.

Python plays an important role in his computational simulations, but also in his industry role as an AI researcher at a new EdTech startup, Rho Zeta AI, where he focuses on innovative applications of Deep Learning for Computer Vision problems for interactive and educational game development.

Hailing from the city of Lagos, Nigeria, Dare is a budding social entrepreneur and OSINT enthusiast and hopes longer-term to return to West Africa and (to?) contribute to (help accelerate?) the development of the Tech industry in his home country.