Analyzing data in real-time gives us the opportunity to derive important insights from data and respond to events as they occur.
More than any other topic, COVID-19 was the most discussed topic globally in 2020. Since the beginning of the outbreak, there have been thousands of articles, and millions of messages broadcasted. Twitter is one of such platforms where you can find endless tweets and mentions of the virus.
The objective of this exercise is to create a real-time visualization dashboard of COVID-19 related tweets using Elasticsearch & Kibana. …
The term “serverless” is becoming very popular in the world of Big Data. The rise of cloud computing is fast democratizing the approach and method by which we store, process, and analyze vast amounts of data. Gone are those days when organizations have to set up large farms of physical servers and storage devices on-premises just to be able to store and process large datasets at speed.
The primary advantage that serverless data analytics architectures offer is the ability to store and analyze a huge amount of data in a highly durable, secure, and scalable manner without the headache of…
FastAPI is a modern and fast Python web framework for building backend applications. FastAPI comes with support for API documentation powered by Swagger, security modules, and type checking to ensure correctness in code.
Why choose FastAPI over other Python web frameworks? FastAPI provides many new, valuable, and convenient features.
Serverless is the way to go when it comes to developing highly scalable and available applications. Serverless does not mean that there is no server, but it means that the tasks associated with infrastructure provisioning and management are completely handled by the cloud service provider and invisible to the developer.
Building applications that are completely serverless enables developers to increase their productivity and bring products to market faster, and it allows organizations to better optimize resources and stay focused on innovation.
Apache Airflow is an open-source workflow automation tool that can be used to programmatically author, schedule, and monitor data processing pipelines. Airflow originated at Airbnb in 2014 and became a top-level open-source project under Apache Software Foundation in January 2019.
Since its inception, Apache Airflow has quickly become the de-facto standard for workflow orchestration. Now, Airflow is in use at more than 200 organizations, including Adobe, Airbnb, Etsy, Google, Lyft, NYC City Planning, Paypal, Reddit, Square, Twitter, and United Airlines, among others.
Apart from its ease to use and the growing community of contributors, what has made Airflow become so…
Provisioning of cloud resources by clicking multiple buttons on the cloud providers’ management console is a time-consuming process that can be automated to achieve faster development timelines and save on operational costs.
In most organizations, deployment of cloud resources is a repetitive process and not a one time task. Imagine a situation where a cloud engineer needs to deploy a complex infrastructure in multiple environments (E.g. dev, test, and prod) or in a multi-cloud situation where you want to replicate the same cloud setup to a different service provider (E.g. replicate GCP to AWS).