SoFunction
Updated on 2024-10-29

A tutorial with examples of Python's Tornado framework for data visualization

Expansion modules used

    xlrd:

Python language , read Excel extension tool . Can be realized in the specified form, the specified cell reading.
Installation is required before use.
Download: /pypi/xlrd
Unzip it and cd to the directory where you unzipped it, and run python install.

    datetime:

Python's built-in modules for manipulating datetime

Functional modules to be implemented

Read xls file and enter into database

Obtaining the day's duty status based on three parameters: year, month and day

Pie chart (number of duty assignments completed for the day/number of duty assignments not completed for the day)

Waterfall chart (all duty officers on duty for the day)

Obtain the duty status for the month based on the year and month parameters

Obtain duty status for the year based on annual parameters

duty system

There are a total of six classes per day:

    8:00 - 9:45
    9:45 - 11:20
    13:30 - 15:10
    15:10 - 17:00
    17:00 - 18:35
    19:00 - 22:00

Each person will work a maximum of one shift per day.

Clocking in is only valid during duty hours and within half an hour before and after.

Clocking in and out of work is required, and lack of clocking in is considered not on duty.

Analyzing Excel spreadsheets

My fingerprint time and attendance machine can export up to one month's worth of punches at a time. One problem is that the month may span two months or a year. For example: 03/21/2015 - 04/20/2015, 12/15/2014 - 01/05/2015. So be sure to watch out for this pitfall when writing your processing methods.

The exported table is shown in the figure:

 (600×375)


=. = It looks like basically no one is on duty, yeah, that's right.
Everyone is so lazy T. T
    Sign...

A brief analysis.

  • The Attendance Record Sheet is the third sheet in the file.
  • The third line has a start and end time
  • The fourth line is a number for all dates
  • Each of the next two rows: the first row contains user information; the second row contains attendance records

reasoning

It was decided to use 3 collections to store the relevant information separately:

  1. user: user information, including id, name, dept.
  2. record: attendance record, including id (user id), y (year), m (month), d (day), check (punch record)
  3. duty: duty schedule, contains id (number of days of the week, example: 1 means Monday), list (list of duty personnel ids), user_id: ["start_time", "end_time"] (user's duty start time and end time)

Read the xls file and store the new attendance records and new users into the database.

According to the year, month and day parameter, query the corresponding record, query the duty arrangement of the day, and match to get the attendance record of the students on duty on the day. Compare the punching time of the duty students with the duty time, determine whether the punching time is normal or not, and calculate the actual duty time and the actual duty percentage.

After that output the data in json format and generate charts using echarts.

Analyzing attendance records for the current month and year is the same thing, though it may be slightly more complicated.

All the explanations and specific ideas are placed in the source code comments, please continue to look down the source code ~!

source code (computing)

   

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import 

import 
import 
import 
import 
import 
import 
from  import define, options

import pymongo
import time
import datetime
import xlrd

define("port", default=8007, help="run on the given port", type=int)

class Application():
  def __init__(self):
    handlers = [
      (r"/", MainHandler),
      (r"/read", ReadHandler),
      (r"/day", DayHandler),
    ]
    settings = dict(
      template_path=((__file__), "templates"),
      static_path=((__file__), "static"),
      debug=True,
      )
    conn = ("localhost", 27017)
     = conn["kaoqin"]
    .__init__(self, handlers, **settings)


class MainHandler():
  def get(self):
    pass

class ReadHandler():
  def get(self):
    #GetCollection
    coll_record = 
    coll_user = 
    #Read excel sheet
    table = xlrd.open_workbook('/Users/ant/Webdev/python/excel/')
    #Read the punch list sheet
    sheet=table.sheet_by_index(2)
    #Read clock-in month range
    row3 = sheet.row_values(2)
    m1 = int(row3[2][5:7])
    m2 = int(row3[2][18:20])
    #Set the current year
    y = int(row3[2][0:4])
    # Set the current month as the first month
    m = m1
    #Read punch date range
    row4 = sheet.row_values(3)
    # Initialize the previous day
    lastday = row4[0]
    # Iterate through the dates in the fourth row
    for d in row4:
      # If the date is less than the previous date
      # Explain that if the month is increased, then the current month will be changed to the second month.
      if d < lastday:
        m = m2
        # If the two current months are December and January respectively
        #States that the year has crossed over, so year +1
        if m1 == 12 and m2 == 1:
          y = y + 1
      # counted by n, ranging from 3 to (total number of rows/2+1)
      # (Total lines/2+1) - 3 = Total users
      # i.e. iterating over all users
      for n in range(3, /2+1):
        # Take the first line of the user, i.e. the user information line
        row_1 = sheet.row_values(n*2-2)
        # Get user id
        u_id = row_1[2]
        # Get user name
        u_name = row_1[10]
        # Access to user departments
        u_dept = row_1[20]
        # Query the user
        user = coll_user.find_one({"id":u_id})
        # Create a new user if the user does not exist in the database
        if not user:
          user = dict()
          user['id'] = u_id
          user['name'] = u_name
          user['dept'] = u_dept
          coll_user.insert(user)
        # Fetch the second line of the user, the attendance record line
        row_2 = sheet.row_values(n*2-1)
        # Get the subscript to change the current date
        idx = (d)
        # Get the current user's attendance record for the current date
        check_data = row_2[idx]
        # Initialize empty attendance record list
        check = list()
        # In groups of 5 characters, iterates through the attendance records and deposits them in the attendance records list
        for i in range(0,len(check_data)/5):
          (check_data[i*5:i*5+5])
        # Query the current user's record for the day
        record = coll_record.find_one({"y":y, "m":m, "d":d, "id":user['id']})
        # Update the record if it exists
        if record:
          for item in check:
            # Add new attendance records to previous records
            if item not in record['check']:
              record['check'].append(item)
              coll_record.save(record)
        # Insert new record if record does not exist
        else:
          record = {"y":y, "m":m, "d":d, "id":user['id'], "check":check}
          coll_record.insert(record)

 

class DayHandler():
  def get(self):
    # Get year, month and day parameters
    y = self.get_argument("y",None)
    m = self.get_argument("m",None)
    d = self.get_argument("d",None)
    # Determine if the parameters are fully set
    if y and m and d:
      # Convert parameters to integers for ease of use
      y = int(y)
      m = int(m)
      d = int(d)
      # Get all records for the day
      coll_record = 
      record = coll_record.find({"y":y, "m":m, "d":d})
      # Get the day of the week
      weekday = (y,m,d).strftime("%w")
      # Get the duty roster for the day
      coll_duty = 
      duty = coll_duty.find_one({"id":int(weekday)})
      # Initialize empty target records (duty officer records for the day)
      target = list()
      # Iterate over all records of the day
      for item in record:
        # When the user of the record has a duty assignment for that day, it is calculated and stored in the target array
        if int(item['id']) in duty['list']:
          # Get the start and end of the user's shift by user id
          start = duty[item['id']][0]
          end = duty[item['id']][1]
          # of hours/seconds of calculated duty
          date1 = (y,m,d,int(start[:2]),int(start[-2:]))
          date2 = (y,m,d,int(end[:2]),int(end[-2:]))
          item['length'] = (date2 - date1).seconds
          # of actual shifts initialized
          item['per'] = 0
          # Initialize clock-in/clock-out times
          item['start'] = 0
          item['end'] = 0
          # Iterate through the user's punch log
          for t in item['check']:
            # When it's earlier than duty time
            if t < start:
              # Calculate the time difference
              date1 = (y,m,d,int(start[:2]),int(start[-2:]))
              date2 = (y,m,d,int(t[:2]),int(t[-2:]))
              dif = (date1 - date2).seconds
              #When clocking in within half an hour before duty time
              if dif <= 1800:
                # Clocked in at work and succeeded
                item['start'] = start
            elif t < end:
              # If you haven't clocked in at work
              if not item['start']:
                # then the current time is recorded as the time to clock in at work
                item['start'] = t
              else:
                # Otherwise record the current time as the off-duty clock-in time
                item['end'] = t
            else:
              # If you've clocked in at work
              if item['start']:
                # Calculate the time difference
                date1 = (y,m,d,int(end[:2]),int(end[-2:]))
                date2 = (y,m,d,int(t[:2]),int(t[-2:]))
                dif = (date1 - date2).seconds
                #When clocking in within half an hour of duty time
                if dif <= 1800:
                  # Clocked in from work
                  item['end'] = end
          # When clocking in and clocking out
          if item['start'] and item['end']:
            # Calculation of actual duty hours
            date1 = (y,m,d,int(item['start'][:2]),int(item['start'][-2:]))
            date2 = (y,m,d,int(item['end'][:2]),int(item['end'][-2:]))
            dif = (date2 - date1).seconds
            # Calculated (actual hours on duty/hours on duty) percentage
            item['per'] = int(dif/float(item['length']) * 100)
          else:
            # Failure to report for work on a regular basis is considered not on duty
            item['start'] = 0
            item['end'] = 0
          # Add records to the target array
          (item)
      # Output data
      ("",
        target = target
        )


def main():
  .parse_command_line()
  http_server = (Application())
  http_server.listen()
  ().start()


if __name__ == "__main__":
  main()

  

{
{% for item in target %}
  {
   'id':{{ item['id'] }},
   'start':{{ item['start'] }},
   'end':{{ item['end'] }},
   'length':{{ item['length'] }}, 
   'per':{{ item['per'] }}
   }
{% end %}
}

ultimate

For the time being, I've only written about reading files and checking the status of a particular day's shift, and I'll continue to finish this little application as previously planned.

I didn't post the test file because it involves the privacy of a bunch of peeps. But if any of you want to actually run it and see what happens, talk to me and I'll send you the file.

A possible database insert statement: ({"id":5, "list":[1,2],1:["19:00", "22:00"],2:["19:00", "22:00"]})

I hope this is helpful to BEGINNERS like me!