Planning tasks for the future is an essential tool for any software developer. While much of the programming we create is intended to respond to explicit triggers or user events, background processes that execute periodically are just as important.
"Results are updated every Monday morning."
"Orders are placed in batches every night."
Even third-party APIs with daily request limits implicitly require this behavior.
"We can only request updates every five minutes."
Fortunately, many smart people have solved this problem and it's not hard to find python native solutions. Advanced Python Scheduler (APS) is a great option with a simple, intuitive API and some of the best documentation in its class.
For this project, we'll focus on integrating the scheduling technology provided by APS with your regular Django application: the Los Angeles Weather app, which periodically polls a third-party weather api for model updates.
The goal is to explore deeper than the Django tutorials while not getting bogged down in any direction.
I. Installation of APS and other dependencies
In your project directory, create a virtual environment and activate it
virtualenv env . env/bin/activate
Install and configure PostgreSQL according to this guide. At this stage, we just need to get SQL Manager up and running on your computer.
Also, I found it helpful to use the PgAdmin PostgreSQL GUI. Details on setting it up on your computer can be found here (using Python3).
Use pip to install all required packages (note that psycopg2 works for PostgreSQL):
pip install apscheduler django psycopg2 requests II. Build your app Create a new Django project: startproject advancedScheduler cd advancedScheduler python startapp weather
In this new directory (the root directory) you will see another folder called advancedScheduler. This is the Django project directory.
To avoid the confusion of having two places with the same name, we will just refer to the "root directory" as the "root directory". Let the following diagram serve as a roadmap for our folder-hopping adventure.
[ super_project_directory/ ] | +----[ env/ ] <-- Virtualenv stuff | +----[ advancedScheduler/ ] <-- the Root Directory | +----[ advancedScheduler/ ] <-- the Django Project Directory | +----[ weather/ ] <-- the Django App Directory
While primarily focused on demonstrating the functionality of the scheduler, let's take a moment to connect the Django application.
We'll start by adding the weather app to the project's INSTALLED_APPS. This is located in the advancedScheduler / file.
INSTALLED_APPS = [ '', '', '', '', '', '', 'weather' ]
Next, add the new URL format to the advancedScheduler / file:
path('', include(''))
Not surprisingly, our next step will be to add the file to the weather app directory. Include the following code in weather /:
from import url from weather import views urlpatterns = [ url(r'^$', .as_view()) ]
Create a template folder in the Weather app directory. Add files to this new folder.
Check out our MTV below.
mould
from import models
from datetime import datetime
class Forecast():
timestamp = ()
temperatue = (max_digits=12,decimal_places=2)
description = (max_length=150)
city = (max_length=150)
def save(self, *args, **kwargs):
if not :
= ()
return super(Forecast, self).save(*args, **kwargs)
Template
<div style="text-align: center">
<h5>Currently in</h5>
<h3>{{city}}</h3>
<h4>{{temperature_in_f}} F | {{temperature_in_c}} C</h4>
<h4>{{desctiprion}}</h4>
<p><em>Last updated {{utc_update_time}} GMT</em></p>
</div>
View
import decimal
from datetime import datetime
from import render
from import TemplateView
from import Forecast
class MainPage(TemplateView):
def get(self, request, **kwargs):
latest_forecast = ('timestamp')
city = latest_forecast.city
temperature_in_c = latest_forecast.temperatue
temperature_in_f = (latest_forecast.temperatue * (1.8)) + 32
description = latest_forecast.
timestamp = "{}/{:02d}/{:02d} - {:02d}:{:02d}:{:02d}".format( t=latest_forecast.timestamp)
return render(
request,
'',
{
'city':city,
'temperature_in_c': temperature_in_c,
'temperature_in_f': round(temperature_in_f,2),
'desctiprion': description,
'utc_update_time': timestamp})
III. Establishing database connections and migrating models
In advancedScheduler /, change the DATABASES value to:
DATABASES = {
'default': {
'ENGINE': '.postgresql_psycopg2',
'NAME': 'advancedScheduler',
'USER': 'some_user_name',
'PASSWORD': 'some_password',
'HOST': 'localhost',
'PORT': '',
}
}
You should know the values for USER, PASSWORD and PORT from the PostgreSQL configuration guides above (here and here).
After establishing a connection with PostgreSQL, it's time to migrate our model. Navigate to the "root directory" and type:
python makemigrations python migrate
With this, our model should be mapped to the database. Go ahead and check everything. Don't worry, I'll be here when you get back.
IV. Forecasting APIs
Time for the fun part. I'm pulling my forecast data from OpenWeatherMap, a free weather API that grants you an access token with a valid email address.
Now, since it is conceptually different from our representation layer, let's create a new ForecastUpdater folder in the root directory. In it, we will add two files: a blank __init__.py file and a file. See the roadmap for reference.
[ super_project_directory/ ]
|
+----[ env/ ]
|
+----[ advancedScheduler/ ] <-- the Root Directory
|
+----[ advancedScheduler/ ]
|
+----[ weather/ ]
|
+----[ forecastUpdater/ ] <-- the new Updater Module
|
+----< __init__.py > <--+
| |-- two new Python files
+----< > <--+
import requests
from import Forecast
def _get_forecast_json():
url = '/data/2.5/weather'
encoded_city_name = 'Los%20Angeles'
country_code = 'us'
access_token = 'your_access_token'
r = ('{0}?q={1},{2}&APPID={3}'.format(
url,
encoded_city_name,
country_code,
access_token))
try:
r.raise_for_status()
return ()
except:
return None
def update_forecast():
json = _get_forecast_json()
if json is not None:
try:
new_forecast = Forecast()
# open weather map gives temps in Kelvin. We want celsius.
temp_in_celsius = json['main']['temp'] - 273.15
new_forecast.temperatue = temp_in_celsius
new_forecast.description = json['weather'][0]['description']
new_forecast.city = json['name']
new_forecast.save()
print("saving...\n" + new_forecast)
except:
pass
There are a few things to keep in mind here. Exception handling is far from robust. Errors are just thrown away-excessive silence is the only sign that something is wrong.
Next, we specify Los Angeles in the code. Configure your server to any desired location.
It's also important to note that update_forecast() takes no arguments. As we will soon see, our advanced python planner has strict no-argument rules. Even methods with a lone self argument won't fly.
V. Advanced Python Planning Programs
We have built the model. We can update the data by calling the API. Now all we need to do is specify how often that API is accessed so that we can provide reasonably up-to-date information without exceeding data access limits.
In the ForecastUpdater module, add a file. Here, we'll use Advanced Python Scheduler to set the cadence of our forecast updates.
OpenWeatherMaps Terms of Use allows 60 calls to be held in an hour to maintain the free tier; updates every five minutes are sufficient.
from datetime import datetime from import BackgroundScheduler from forecastUpdater import forecastApi def start(): scheduler = BackgroundScheduler() scheduler.add_job(forecastApi.update_forecast, 'interval', minutes=5) ()
This is probably the simplest implementation of APS you can find. If you check out their website or the few working examples on GitHub, then you will find a full toolbox of features and settings that you can use to time it to be as detailed as possible.
After configuring the scheduler the way we want it, we can connect it to the Django application.
Ideally, we want to press play once on a scheduler and then have it perform its task. We need a consistent and reliable way to initialize the schedule once and only once. For us, Django is exactly the place for this type of runtime initialization logic.
In the weather / file, you'll find a stub for a class called WeatherConfig, which inherits from Django's AppConfig class.
class WeatherConfig(AppConfig): name = 'weather'
To let Django know that it needs to start the updater on startup, we override the () method.
from import AppConfig class WeatherConfig(AppConfig): name = 'weather' def ready(self): from forecastUpdater import updater ()
It's important to remember that due to the complexity of inheritance, any import of this override must be located within the body of the ready() method. Django also warns against interacting directly with the database in our override; production, debugging, rain or shine, will execute this code every time the weather application is launched.
Finally, we now need to update the INSTALLED_APPS variable again in advancedScheduler /. Django needs to know that we're going to use a custom configuration to run the weather app.
INSTALLED_APPS = [ '', '', '', '', '', '', '' ]
Put it all together.
Only. At this point, we can start our application and then let the updater perform its actions.
python runserver --noreload
The -noreload flag prevents Django from launching a second instance of the weather app - this is the default behavior in debug mode. The second instance means that all our scheduled tasks will trigger twice.
Initially, our results looked incomplete. Since we scheduled the updater logic to run every five minutes, we kept twitching ...... To keep things interesting, it might be wise to shorten the interval between prudent refreshes, or call update_forecast() once at initialization time.
Seven. Final thoughts
We did it! Our weather app is ready to be shared with the world (check out my info here).
Advanced Python Scheduler is a great tool that any Python developer knows about. It hides the complexity of very common business requirements behind an intuitive API. Consider that the installer uses only three lines of code.
The real trick of the project is interacting with the Django framework-configuring, migrating, initializing. Then task automation becomes an afterthought. In five minutes you are done.
/kmhoran/la-weather-app
The above is a small introduction to the use of apscheduler in django in the implementation of scheduled tasks , I hope to help you !