Share bits and pieces of learning Python crawler, data analysis, data mining.
I recently discovered a way to download Shakeology watermark-free videos.
# Jitterbug interface
The "url" parameter value is the link copied from Jitterbug.
Python Download
First, take a look at the results you get by visiting the Shakeology link directly.
Properly watermarked...
Next, open your browser's developer tools and look at the address of the video.
You can find that "playAddr" is the address of the video, copy it and access it.
The link redirects to a link that starts with "v9", however there is still a watermark.
Next is the point, first you need to enable your browser to modify the UA, which is often used by crawlers as "User-Agent".
I'm using a Mac + Google Chrome, so let's talk about how I modified it myself.
For Windows, please use Baidu.
First create a folder in your computer's manuscript.
The path to this folder is as follows.
/Users/star-river/Documents/MyChrome
And run the following code from a terminal in the root directory.
open -n /Applications/Google\ / --args --disable-web-security --user-data-dir=/Users/star-river/Documents/MyChrome
I was able to change the UA of my Google Chrome browser!
Still visiting the Jitterbug link directly, you can see that the results are not the same as they were initially.
Just look up the interface in this mode.
It was found that this interface starting with "?item_ids" contains the watermark-free Shake Shack video we want.
It's those two links in the list under "play_addr".
The interface that starts with "?item_ids" has two parameters that we need to get in another interface.
In this way, we know the values of the "item_ids" and "dytk" parameters.
But those two links we get by visiting them directly with a browser won't bring up the video directly, it needs to be the same as above.
Change the UA as well, the link here will fail if it's still accessed with the UA "iPhone X".
For what reason, Little F doesn't know...
Change your browser UA to "Responsive" to access it and the link will redirect.
This way the watermark-free Shake Shack video is done.
But it would be a pain in the ass if every video needed to be downloaded that way.
So wrote the code that allows you to download the video using Python.
import requests
import json
import re
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/ajpg,*/*;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8',
'cache-control': 'max-age=0',
# This seems to be important #
'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Mobile Safari/537.36',
}
def download(url):
"""
Download Shake Shack watermark-free videos
"""
# Get interface parameters
html = (url=url, headers=headers)
title = ('itemId: "(.*?)",', )[0]
dytk = ('dytk: "(.*?)" }', )[0]
# Splice interface
url_item = '/web/api/v2/aweme/iteminfo/?item_ids=' + title + '&dytk=' + dytk
# Get Shake Shack watermark-free video links
html_item = (url=url_item, headers=headers)
# String to Dictionary
content = (html_item.text)
# Video interface
url_video = content['item_list'][0]['video']['play_addr']['url_list'][1]
response = (url_video, headers=headers, allow_redirects=True)
# Get redirected links, this is also the download link of the video without watermarks, but this is not used
redirect =
print(redirect)
# The video is binary, it needs to be downloaded this way #
video = (url_video, headers=headers).content
video_name = "douyin.mp4"
with open(video_name, 'wb') as f:
(video)
()
print("Download complete.")
if __name__ == '__main__':
# Jitterbug links
url = '/XJj85H/'
download(url)
Watermark-free video perfect download.
Interface Download
Now that you know how to download videos in Python.
Then Little F wanted to make it a little easier for people to download, so he deployed the program to the server.
You just need to download the video through the interface of Little F. The code is as follows.
from flask import Flask, request, send_file
import requests
import json
import re
app = Flask(__name__)
# Accept get method access only
@("/douyin/", methods=["GET"])
def check():
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/ajpg,*/*;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8',
'cache-control': 'max-age=0',
'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Mobile Safari/537.36',
}
# Default return content
return_dict = {'code': 1, 'result': False, 'msg': 'Request successful'}
# Determine if the input parameter is null
if is None:
return_dict['return_code'] = '504'
return_dict['return_info'] = 'Request parameter is empty'
return (return_dict, ensure_ascii=False)
# Get incoming parameters
get_data = .to_dict()
url = get_data.get('url')
# Get interface parameters
html = (url=url, headers=headers)
title = ('itemId: "(.*?)",', )[0]
dytk = ('dytk: "(.*?)" }', )[0]
# Splice interface
url_item = '/web/api/v2/aweme/iteminfo/?item_ids=' + title + '&dytk=' + dytk
# Get Shake Shack watermark-free video links
html_item = (url=url_item, headers=headers)
# String to Dictionary
content = (html_item.text)
# Getting video-related information
# data = {}
# Description of the video
# data['videoDesc'] = content['item_list'][0]['desc']
# Cover art, small art for the video #
# data['dynamiCoverUrl'] = content['item_list'][0]['video']['dynamic_cover']['url_list'][0]
# Cover art for the video, big picture #
# data['staticCoverUrl'] = content['item_list'][0]['video']['origin_cover']['url_list'][0]
# of comments on the video
# data['comments'] = content['item_list'][0]['statistics']['comment_count']
# of likes on the video
# data['prise'] = content['item_list'][0]['statistics']['digg_count']
# Video interface
url_video = content['item_list'][0]['video']['play_addr']['url_list'][1]
response = (url_video, headers=headers, allow_redirects=True)
# Get redirected links, this is also the download link of the video without watermarks, but this is not used
redirect =
# print(redirect)
# Download link for the video
# data['videoPlayAddr'] = redirect
# Return information about the video
# return_dict['result'] = data
# Return results
# return (return_dict, ensure_ascii=False)
video = (url=redirect, headers=headers).content
video_name = "douyin.mp4"
with open(video_name, 'wb') as f:
(video)
()
return send_file('douyin.mp4')
if __name__ == "__main__":
# Local debugging
(debug=True)
# Deployment goes live
# (host='127.0.0.1', port=443)
If Flask and the Requests library are installed locally, this program can be run directly.
And you will be able to download the watermark-free Shakeology videos you want.
# Local interfaces
http://127.0.0.1:500/douyin/?url=/CoQBx1/
If deployed to a server, port 443 is required.
This article on Python download Shake Shack watermark-free video is introduced to this article, more related Python download Shake Shack watermark-free video content, please search for my previous articles or continue to browse the following related articles I hope you will support me more in the future!