We know that in the Android system, a lightweight logging system is provided. This logging system is implemented in the kernel space in the form of a driver, and the Java interface and C/C++ interface are provided in the user space to use this logging system, depending on whether you are writing an Android application or a system component. In the previous article briefly discussing the use of LOG in Android system development, we have briefly introduced the usage of Log in Android application development. In this article, we will further analyze the source code of the Logger driver, so that we have a deep understanding of the Android log system.
Since the Android log system is implemented in the kernel space in the form of a driver, we need to obtain the Android kernel source code for analysis. Please refer to the previous two articles on downloading, compiling and installing the latest Android source code on Ubuntu and downloading, compiling and installing the latest Android kernel source code (Linux Kernel) on Ubuntu to download the Android source code project. The Logger driver is mainly composed of two files, namely:
kernel/common/drivers/staging/android/
kernel/common/drivers/staging/android/
Next, we will introduce the relevant data structure of the Logger driver, and then conduct scenario analysis on the Logger driver source code, respectively, log system initialization scenario, log reading scenario, and log writing scenario.
1. Relevant data structure of the Logger driver.
Let’s first look at the contents of the header file:
#ifndef _LINUX_LOGGER_H
#define _LINUX_LOGGER_H
#include <linux/>
#include <linux/>
struct logger_entry {
__u16 len; /* length of the payload */
__u16 __pad; /* no matter what, we get 2 bytes of padding */
__s32 pid; /* generating process's pid */
__s32 tid; /* generating process's tid */
__s32 sec; /* seconds since Epoch */
__s32 nsec; /* nanoseconds */
char msg[0]; /* the entry's payload */
};
#define LOGGER_LOG_RADIO "log_radio" /* radio-related messages */
#define LOGGER_LOG_EVENTS "log_events" /* system/hardware events */
#define LOGGER_LOG_MAIN "log_main" /* everything else */
#define LOGGER_ENTRY_MAX_LEN (4*1024)
#define LOGGER_ENTRY_MAX_PAYLOAD \
(LOGGER_ENTRY_MAX_LEN - sizeof(struct logger_entry))
#define __LOGGERIO 0xAE
#define LOGGER_GET_LOG_BUF_SIZE _IO(__LOGGERIO, 1) /* size of log */
#define LOGGER_GET_LOG_LEN _IO(__LOGGERIO, 2) /* used log len */
#define LOGGER_GET_NEXT_ENTRY_LEN _IO(__LOGGERIO, 3) /* next entry len */
#define LOGGER_FLUSH_LOG _IO(__LOGGERIO, 4) /* flush log */
#endif /* _LINUX_LOGGER_H */
struct logger_entry is a structure used to describe a log record. The len member variable records the length of the payload for this record, the length of the log record itself specified by the payload, but does not include the struct logger_entry structure used to describe the record. Recall that when we call the interface to use the log system, we will specify the priority levels of the log Priority, Tag string and Msg string. The lengths of Priority + Tag + Msg add up to the payload length of the record. __pad member variable is used to align structures. The pid and tid member variables are used to record which process wrote this record. The sec and nsec member variables record the time the log is written. The msg member variable records the content of the payload, and its size is determined by the len member variable.
Next, define two macros:
#define LOGGER_ENTRY_MAX_LEN (4*1024)
#define LOGGER_ENTRY_MAX_PAYLOAD \ (LOGGER_ENTRY_MAX_LEN - sizeof(struct logger_entry))
From these two macros, it can be seen that the payload length of each log record plus the length of the structure logger_entry cannot exceed 4K bytes.
Other macros are also defined in the file, which readers can analyze by themselves. In the following analysis, we will also explain in detail when encountering it.
Let’s look at the definitions of other related data structures in the file:
/*
* struct logger_log - represents a specific log, such as 'main' or 'radio'
*
* This structure lives from module insertion until module removal, so it does
* not need additional reference counting. The structure is protected by the
* mutex 'mutex'.
*/
struct logger_log {
unsigned char * buffer; /* the ring buffer itself */
struct miscdevice misc; /* misc device representing the log */
wait_queue_head_t wq; /* wait queue for readers */
struct list_head readers; /* this log's readers */
struct mutex mutex; /* mutex protecting buffer */
size_t w_off; /* current write head offset */
size_t head; /* new readers start here */
size_t size; /* size of the log */
};
/*
* struct logger_reader - a logging device open for reading
*
* This object lives from open to release, so we don't need additional
* reference counting. The structure is protected by log->mutex.
*/
struct logger_reader {
struct logger_log * log; /* associated log */
struct list_head list; /* entry in logger_log's list */
size_t r_off; /* current read head offset */
};
/* logger_offset - returns index 'n' into the log via (optimized) modulus */
#define logger_offset(n) ((n) & (log->size - 1))
The structure struct logger_log is the place that is really used to save logs. The buffer member variable is a memory buffer that stores log information, and its size is determined by the size member variable. From the misc member variable, it can be seen that the device used by the logger driver belongs to the misc type device. By executing the cat /proc/devices command on the Android emulator (refer to the article downloading, compiling and installing Android's latest kernel source code (Linux Kernel) on Ubuntu), it can be seen that the master device number of the misc type device is 10. For relevant knowledge about the main device number, you can refer to the book Linux Driver Development mentioned in the Android Learning Startup article. The wq member variable is a waiting queue that holds the process waiting to read the log. The readers member variable is used to save the process currently reading the log. The process that is reading the log is described by the structure logger_reader. The mutex member variable is a mutex that protects the concurrent access of the log. It can be seen that the reading and writing problem of the log system here is actually a producer-consumer problem, so mutexes are needed to protect the concurrent access of the log. The w_off member variable is used to record where the next log should start writing. The head member variable is used to indicate where to start reading the log in the log file.
The structure struct logger_reader is used to represent a process that reads the log, and the log member variable points to the log buffer to be read. List member variable is used to connect other readers processes. The r_off member variable indicates the location of the currently read log in the buffer.
The memory buffer buffer used in the struct logger_log structure used to save log information is a circular buffer used in a circular manner. The content saved in the buffer is in struct logger_entry units, and the composition of each unit is:
struct logger_entry | priority | tag | msg
Since the memory buffer is a loop buffer used by a loop, given an offset value, its position in the buffer is determined by the lower logger_offset:
#define logger_offset(n) ((n) & (log->size - 1))
2. Analysis of the initialization process of the Logger driver module.
Continue to look at the file and define three log devices:
/*
* Defines a log structure with name 'NAME' and a size of 'SIZE' bytes, which
* must be a power of two, greater than LOGGER_ENTRY_MAX_LEN, and less than
* LONG_MAX minus LOGGER_ENTRY_MAX_LEN.
*/
#define DEFINE_LOGGER_DEVICE(VAR, NAME, SIZE) \
static unsigned char _buf_ ## VAR[SIZE]; \
static struct logger_log VAR = { \
.buffer = _buf_ ## VAR, \
.misc = { \
.minor = MISC_DYNAMIC_MINOR, \
.name = NAME, \
.fops = &logger_fops, \
.parent = NULL, \
}, \
.wq = __WAIT_QUEUE_HEAD_INITIALIZER(VAR .wq), \
.readers = LIST_HEAD_INIT(VAR .readers), \
.mutex = __MUTEX_INITIALIZER(VAR .mutex), \
.w_off = 0, \
.head = 0, \
.size = SIZE, \
};
DEFINE_LOGGER_DEVICE(log_main, LOGGER_LOG_MAIN, 64*1024)
DEFINE_LOGGER_DEVICE(log_events, LOGGER_LOG_EVENTS, 256*1024)
DEFINE_LOGGER_DEVICE(log_radio, LOGGER_LOG_RADIO, 64*1024)
They are log_main, log_events and log_radio, respectively, with names LOGGER_LOG_MAIN, LOGGER_LOG_EVENTS and LOGGER_LOG_RADIO respectively. Their secondary device number is MISC_DYNAMIC_MINOR, that is, dynamic allocation during registration. In the file, there are definitions of these three macros:
#define LOGGER_LOG_RADIO "log_radio" /* radio-related messages */
#define LOGGER_LOG_EVENTS "log_events" /* system/hardware events */
#define LOGGER_LOG_MAIN "log_main" /* everything else */
The comments illustrate the purpose of these three logging devices. The registered log device file operation method is logger_fops:
static struct file_operations logger_fops = {
.owner = THIS_MODULE,
.read = logger_read,
.aio_write = logger_aio_write,
.poll = logger_poll,
.unlocked_ioctl = logger_ioctl,
.compat_ioctl = logger_ioctl,
.open = logger_open,
.release = logger_release,
};
The initialization function of the log driver module is logger_init:
static int __init logger_init(void)
{
int ret;
ret = init_log(&log_main);
if (unlikely(ret))
goto out;
ret = init_log(&log_events);
if (unlikely(ret))
goto out;
ret = init_log(&log_radio);
if (unlikely(ret))
goto out;
out:
return ret;
}
device_initcall(logger_init);
The logger_init function initializes the above-mentioned three log devices by calling the init_log function:
static int __init init_log(struct logger_log *log)
{
int ret;
ret = misc_register(&log->misc);
if (unlikely(ret)) {
printk(KERN_ERR "logger: failed to register misc "
"device for log '%s'!\n", log->);
return ret;
}
printk(KERN_INFO "logger: created %luK log '%s'\n",
(unsigned long) log->size >> 10, log->);
return 0;
}
The init_log function mainly calls the misc_register function to register the misc device. The misc_register function is defined in the kernel/common/drivers/char/file:
* misc_register - register a miscellaneous device
* @misc: device structure
*
* Register a miscellaneous device with the kernel. If the minor
* number is set to %MISC_DYNAMIC_MINOR a minor number is assigned
* and placed in the minor field of the structure. For other cases
* the minor number requested is used.
*
* The structure passed is linked into the kernel and may not be
* destroyed until it has been unregistered.
*
* A zero is returned on success and a negative errno code for
* failure.
*/
int misc_register(struct miscdevice * misc)
{
struct miscdevice *c;
dev_t dev;
int err = 0;
INIT_LIST_HEAD(&misc->list);
mutex_lock(&misc_mtx);
list_for_each_entry(c, &misc_list, list) {
if (c->minor == misc->minor) {
mutex_unlock(&misc_mtx);
return -EBUSY;
}
}
if (misc->minor == MISC_DYNAMIC_MINOR) {
int i = DYNAMIC_MINORS;
while (--i >= 0)
if ( (misc_minors[i>>3] & (1 << (i&7))) == 0)
break;
if (i<0) {
mutex_unlock(&misc_mtx);
return -EBUSY;
}
misc->minor = i;
}
if (misc->minor < DYNAMIC_MINORS)
misc_minors[misc->minor >> 3] |= 1 << (misc->minor & 7);
dev = MKDEV(MISC_MAJOR, misc->minor);
misc->this_device = device_create(misc_class, misc->parent, dev, NULL,
"%s", misc->name);
if (IS_ERR(misc->this_device)) {
err = PTR_ERR(misc->this_device);
goto out;
}
/*
* Add it to the front, so that later devices can "override"
* earlier defaults
*/
list_add(&misc->list, &misc_list);
out:
mutex_unlock(&misc_mtx);
return err;
}
After the registration is completed, create the device file node through device_create. Here, three device files /dev/log/main, /dev/log/events and /dev/log/radio will be created, so that the user space can interact with the driver by reading and writing these three files.
3. Analysis of logging reading process of Logger driver.
Continue to look at the file. The registered method to read the log device file is logger_read:
/*
* logger_read - our log's read() method
*
* Behavior:
*
* - O_NONBLOCK works
* - If there are no log entries to read, blocks until log is written to
* - Atomically reads exactly one log entry
*
* Optimal read size is LOGGER_ENTRY_MAX_LEN. Will set errno to EINVAL if read
* buffer is insufficient to hold next entry.
*/
static ssize_t logger_read(struct file *file, char __user *buf,
size_t count, loff_t *pos)
{
struct logger_reader *reader = file->private_data;
struct logger_log *log = reader->log;
ssize_t ret;
DEFINE_WAIT(wait);
start:
while (1) {
prepare_to_wait(&log->wq, &wait, TASK_INTERRUPTIBLE);
mutex_lock(&log->mutex);
ret = (log->w_off == reader->r_off);
mutex_unlock(&log->mutex);
if (!ret)
break;
if (file->f_flags & O_NONBLOCK) {
ret = -EAGAIN;
break;
}
if (signal_pending(current)) {
ret = -EINTR;
break;
}
schedule();
}
finish_wait(&log->wq, &wait);
if (ret)
return ret;
mutex_lock(&log->mutex);
/* is there still something to read or did we race? */
if (unlikely(log->w_off == reader->r_off)) {
mutex_unlock(&log->mutex);
goto start;
}
/* get the size of the next entry */
ret = get_entry_len(log, reader->r_off);
if (count < ret) {
ret = -EINVAL;
goto out;
}
/* get exactly one entry from the log */
ret = do_read_log_to_user(log, reader, buf, ret);
out:
mutex_unlock(&log->mutex);
return ret;
}
Notice, at the beginning of the function, the struct logger_reader that indicates the reading log context is saved in the private_data member variable of the file pointer. This is set when opening the device file. The device file opening method islogger_open：
/*
* logger_open - the log's open() file operation
*
* Note how near a no-op this is in the write-only case. Keep it that way!
*/
static int logger_open(struct inode *inode, struct file *file)
{
struct logger_log *log;
int ret;
ret = nonseekable_open(inode, file);
if (ret)
return ret;
log = get_log_from_minor(MINOR(inode->i_rdev));
if (!log)
return -ENODEV;
if (file->f_mode & FMODE_READ) {
struct logger_reader *reader;
reader = kmalloc(sizeof(struct logger_reader), GFP_KERNEL);
if (!reader)
return -ENOMEM;
reader->log = log;
INIT_LIST_HEAD(&reader->list);
mutex_lock(&log->mutex);
reader->r_off = log->head;
list_add_tail(&reader->list, &log->readers);
mutex_unlock(&log->mutex);
file->private_data = reader;
} else
file->private_data = log;
return 0;
When the log device file is newly opened, the log is read from the log->head position and saved in the member variable r_off of the struct logger_reader.
The while loop at the start label is waiting for the log to be read. If there is no new log to be read, then the process to read will enter a dormant state, wait for the new log to be written before wake-up. This is achieved through the two calls prepare_wait and schedule. If there is no new log to read and the device file is not opened in a non-blocking O_NONBLOCK mode or there is a signal to be processed (signal_pending(current)), then return directly and no longer wait for the new log to be written. The method to determine whether there is a new log currently available is:
ret = (log->w_off == reader->r_off);
That is, determine whether the write position of the current buffer and the reading position of the current read process are equal. If it is not equal, it means that there is a new log to read.
Continue to look down. If there is a new log to read, then first use get_entry_len to get the length of the next readable log record. From here, we can see that the log reading process is read in the unit of log records, and only one record is read at a time. The function of get_entry_len is implemented as follows:
/*
* get_entry_len - Grabs the length of the payload of the next entry starting
* from 'off'.
*
* Caller needs to hold log->mutex.
*/
static __u32 get_entry_len(struct logger_log *log, size_t off)
{
__u16 val;
switch (log->size - off) {
case 1:
memcpy(&val, log->buffer + off, 1);
memcpy(((char *) &val) + 1, log->buffer, 1);
break;
default:
memcpy(&val, log->buffer + off, 2);
}
return sizeof(struct logger_entry) + val;
As we mentioned above, each log record consists of two parts, one is the structure struct logger_entry used to describe this log record, and the other is the record body itself, that is, the payload. The length of the structure struct logger_entry is fixed. As long as you know the length of the payload, you can know the length of the entire log record. The length of the payload is recorded in the member variable len of the struct logger_entry structure, and the address of the len member variable is the same as the address of the struct logger_entry, so you only need to read two bytes at the beginning of the record. Since the logging buffer is used in a loop, these two bytes may be the first byte stored in the last byte of the buffer, and the second byte stored in the first section of the buffer. In addition, these two bytes are connected together. Therefore, consider it in two situations. For the former, the payload length of the log record is obtained by reading the last byte and the first byte of the buffer, and for the latter, the value of two consecutive bytes is directly read into the local variable val. These two cases are distinguished by judging the difference between the size of the log buffer and the position of the log record to be read in the buffer. If the difference is 1, it means that it is the former case. Finally, adding the length of the payload val to the length of the struct logger_entry to get the total length of the log record to be read.
Then look down and get the length of the record to be read, call the do_read_log_to_user function to perform the real read action:
static ssize_t do_read_log_to_user(struct logger_log *log,
struct logger_reader *reader,
char __user *buf,
size_t count)
{
size_t len;
/*
* We read from the log in two disjoint operations. First, we read from
* the current read head offset up to 'count' bytes or to the end of
* the log, whichever comes first.
*/
len = min(count, log->size - reader->r_off);
if (copy_to_user(buf, log->buffer + reader->r_off, len))
return -EFAULT;
/*
* Second, we read any remaining bytes, starting back at the head of
* the log.
*/
if (count != len)
if (copy_to_user(buf + len, log->buffer, count - len))
return -EFAULT;
reader->r_off = logger_offset(reader->r_off + count);
return count;
}
This function simply calls the copy_to_user function to copy the content specified in the log buffer in the kernel space to the memory buffer in the user space. At the same time, the read offset r_off in the context information of the current read log process is advanced to the beginning of the next log record.
4. Analysis of logging writing process of Logger driver.
Continue to read the file. The registered method to write to the log device file islogger_aio_write：
/*
* logger_aio_write - our write method, implementing support for write(),
* writev(), and aio_write(). Writes are our fast path, and we try to optimize
* them above all else.
*/
ssize_t logger_aio_write(struct kiocb *iocb, const struct iovec *iov,
unsigned long nr_segs, loff_t ppos)
{
struct logger_log *log = file_get_log(iocb->ki_filp);
size_t orig = log->w_off;
struct logger_entry header;
struct timespec now;
ssize_t ret = 0;
now = current_kernel_time();
= current->tgid;
= current->pid;
= now.tv_sec;
= now.tv_nsec;
= min_t(size_t, iocb->ki_left, LOGGER_ENTRY_MAX_PAYLOAD);
/* null writes succeed, return zero */
if (unlikely(!))
return 0;
mutex_lock(&log->mutex);
/*
* Fix up any readers, pulling them forward to the first readable
* entry after (what will be) the new write offset. We do this now
* because if we partially fail, we can end up with clobbered log
* entries that encroach on readable buffer.
*/
fix_up_readers(log, sizeof(struct logger_entry) + );
do_write_log(log, &header, sizeof(struct logger_entry));
while (nr_segs-- > 0) {
size_t len;
ssize_t nr;
/* figure out how much of this vector we can keep */
len = min_t(size_t, iov->iov_len, - ret);
/* write out this segment's payload */
nr = do_write_log_from_user(log, iov->iov_base, len);
if (unlikely(nr < 0)) {
log->w_off = orig;
mutex_unlock(&log->mutex);
return nr;
}
iov++;
ret += nr;
}
mutex_unlock(&log->mutex);
/* wake up any blocked readers */
wake_up_interruptible(&log->wq);
return ret;
The input parameter iocb represents the io context, iov represents the content to be written, and the length is nr_segs, which means that the content with nr_segs segments is to be written. We know that the structure of each log to be written is:
struct logger_entry | priority | tag | msg
Among them, the contents of the three segments: priority, tag and msg are passed from the user space by the iov parameters, corresponding to the three elements in the iov respectively. logger_entry is constructed from kernel space:
struct logger_entry header;
struct timespec now;
now = current_kernel_time(); = current->tgid;
= current->pid;
= now.tv_sec;
= now.tv_nsec;
= min_t(size_t, iocb->ki_left, LOGGER_ENTRY_MAX_PAYLOAD);
Then calldo_write_logFirst write the logger_entry structure into the log buffer:
/*
* do_write_log - writes 'len' bytes from 'buf' to 'log'
*
* The caller needs to hold log->mutex.
*/
static void do_write_log(struct logger_log *log, const void *buf, size_t count)
{
size_t len;
len = min(count, log->size - log->w_off);
memcpy(log->buffer + log->w_off, buf, len);
if (count != len)
memcpy(log->buffer, buf + len, count - len);
log->w_off = logger_offset(log->w_off + count);
Since logger_entry is allocated by the kernel stack space, just copy it with memcpy.
Then, the content of iov is written into the log buffer through a while loop, that is, the priority level of the log, the log Tag and the log body Msg:
while (nr_segs-- > 0) {
size_t len;
ssize_t nr;
/* figure out how much of this vector we can keep */
len = min_t(size_t, iov->iov_len, - ret);
/* write out this segment's payload */
nr = do_write_log_from_user(log, iov->iov_base, len);
if (unlikely(nr < 0)) {
log->w_off = orig;
mutex_unlock(&log->mutex);
return nr;
}
iov++;
ret += nr;
}
Here, we also missed an important step:
/* * Fix up any readers, pulling them forward to the first readable * entry after (what will be) the new write offset. We do this now * because if we partially fail, we can end up with clobbered log * entries that encroach on readable buffer. */ fix_up_readers(log, sizeof(struct logger_entry) + );
Why do we need to call the fix_up_reader function? What is this function for? This is true, since the log buffer is used in a loop, that is, if the old log records are not read in time and the contents of the buffer have been used up, the old records need to be overwritten to accommodate the new records. The content that will be overwritten may be the location of the log to be read next time for some readers, and the location of the logs prepared for the new readers starting to read. Therefore, these positions need to be adjusted so that they can point to a new and valid position. Let's take a look at the implementation of the fix_up_reader function:
/*
* fix_up_readers - walk the list of all readers and "fix up" any who were
* lapped by the writer; also do the same for the default "start head".
* We do this by "pulling forward" the readers and start head to the first
* entry after the new write head.
*
* The caller needs to hold log->mutex.
*/
static void fix_up_readers(struct logger_log *log, size_t len)
{
size_t old = log->w_off;
size_t new = logger_offset(old + len);
struct logger_reader *reader;
if (clock_interval(old, new, log->head))
log->head = get_next_entry(log, log->head, len);
list_for_each_entry(reader, &log->readers, list)
if (clock_interval(old, new, reader->r_off))
reader->r_off = get_next_entry(log, reader->r_off, len);
}
judgelog->headAnd all readers reader reader's current read offset reader->r_off is in the covered area, if so, you need to call get_next_entry to get the starting position of the next valid record to adjust the current position:
/*
* get_next_entry - return the offset of the first valid entry at least 'len'
* bytes after 'off'.
*
* Caller must hold log->mutex.
*/
static size_t get_next_entry(struct logger_log *log, size_t off, size_t len)
{
size_t count = 0;
do {
size_t nr = get_entry_len(log, off);
off = logger_offset(off + nr);
count += nr;
} while (count < len);
return off;
}
And determining whether the current read offset reader->r_off of log->head and all reader reader is within the covered area, it is implemented through the clock_interval function:
/*
* clock_interval - is a < c < b in mod-space? Put another way, does the line
* from a to b cross c?
*/
static inline int clock_interval(size_t a, size_t b, size_t c)
{
if (b < a) {
if (a < c || b >= c)
return 1;
} else {
if (a < c && b >= c)
return 1;
}
return 0;
}
Finally, after the log is written, the reader process waiting for the new log needs to be awakened:
/* wake up any blocked readers */
wake_up_interruptible(&log->wq);
At this point, the main logic of the Logger driver has been analyzed, and there are some other interfaces, such as logger_poll, logger_ioctl and logger_release functions, which are relatively simple and can be analyzed by reading. One thing to mention here is that since the Logger driver module will not be uninstalled when exiting the system, this module does not have the module_exit function, and the reference counting technology is not used for the objects defined in the module.
This article focuses on the implementation of the Android log system in the kernel space. In the next article, we will continue to introduce the implementation process of the Java and C/C++ LOG calling interfaces provided to Android applications in user space.
The above is a compilation of the Logger source code information in Android source code. We will continue to add relevant information in the future. Thank you for your support for this website!