1. Preface
There are many Android phones on the market now that support tapping the screen unlocking. Taping the screen unlocking is a very practical function, but on the one hand, it only supports tapping the screen, and on the other hand, it can only be used to unlock or lock the screen. Moreover, our application developers cannot get in and cannot play at all. Developers, developers, since we are developers, why don’t we do some big news? Then this time I will teach you how to use code to realize the tap recognition of your mobile phone. Doesn’t it sound very interesting? I’m a little eager to try it out. In fact, there are already applications that implement this function on iOS: Knock, an app that knocks to unlock Mac computers, priced at US$4.99, about RMB 33. Sometimes I really want to do iOS development and can happily price my application and make extra money happily. Back to the topic, since ios can be implemented, we naturally cannot fall behind on Android. Now I will lead you to use code to realize the tap recognition of your mobile phone.
This article uses Java as the example language and Android as the example platform.
2. Functional implementation
2.1. Implementation ideas
Speaking of tap recognition, what would you consider using to implement it, sensor? Yes, that's right, as the only way to recognize gestures in mobile phones, we naturally need to use sensors to identify taps, but there are many types of Android sensors, which one should we choose?
In the era of Android 2.3, the Android system has defined 11 sensors. In the era of Android 6.0, the number of sensors defined by the system has reached 26. Which one do we use to use so many sensors? In fact, we only need to consider the 11 sensors provided by the 2.3 era. On the one hand, the sensor parts added later, such as heartbeat sensors, require hardware support, which many mobile phones cannot support such sensors. On the other hand, the 11 sensors in the 2.3 era are already quite powerful and can support the recognition of most gestures and postures. So now I will list the above 11 sensors:
SENSOR_TYPE_ACCELEROMETER Acceleration SENSOR_TYPE_MAGNETIC_FIELD magnetic force SENSOR_TYPE_ORIENTATION direction SENSOR_TYPE_GYROSCOPE Gyro SENSOR_TYPE_LIGHT Light sensing SENSOR_TYPE_PRESSURE pressure SENSOR_TYPE_TEMPERATURE temperature SENSOR_TYPE_PROXIMITY near SENSOR_TYPE_GRAVITY gravity SENSOR_TYPE_LINEAR_ACCELERATION 线性Acceleration SENSOR_TYPE_ROTATION_VECTOR Rotating vector
For detailed descriptions of these 11 sensors, you can go tohttps:///article/Check it out, in fact, I have always suspected that the LG G3's tap unlocking is related to the light sensor or proximity sensor, because I was unable to tap unlocking when I was suspended directly above the head of the LG G3. After removing it, I returned to normal. The tap lock screen should only be related to the touch screen, because no matter how I block the sensor, the tap lock screen function is completely unaffected.
Back to the topic, after understanding these 11 sensors, which sensors do we need to choose to implement the function? Let’s simulate the phone tapping situation, place the phone flat on the desktop, and when the finger taps the phone, the finger gives the phone a force, and the desktop gives the phone a reaction force. Considering that the desktop does not deform, the phone’s force balance acceleration is 0, but will the phone’s acceleration sensor data change at this time? The answer is yes, the phone’s acceleration sensor data will have a short but obvious change. Why? The phone’s force balance acceleration is 0 because it is a whole, but the internal components will still be influenced by complex forces between each other. It is not a force balance at the same time. In fact, a different idea is. Use a container with a smooth shape similar to the mobile phone. Put a few glass balls in the container and tap it a few times. The container will not move, but has the glass ball moved? Although the components inside the mobile phone are much more stable than glass balls, they must follow the Basic Law and accept the role of honestly.
The above scenario is a scene placed flat on the desktop. The actual life scenarios are often more complex and diverse. However, no matter which scenario, there is no doubt that the tapping operation of the mobile phone should lead to obvious changes in the data transmitted by the acceleration sensor. So we now understand what sensor should be selected as our tap recognition tool. However, there are two acceleration sensors, acceleration sensor and linear acceleration sensor. Which one should we choose? The data provided by the acceleration sensor is the acceleration of the mobile phone under the influence of gravity. The data provided by the linear acceleration sensor is the acceleration of the mobile phone that excludes gravity. It can intuitively reflect the stress condition of the mobile phone after excluding gravity. It is very suitable for tapping recognition. So should we choose a linear acceleration sensor? On the contrary, we need to choose an acceleration sensor. The linear acceleration sensor provided by Android is based on software. Different platforms may not handle the linear acceleration sensor the same. In fact, when tapping Samsung S4, LG On the back of one model in G3, there is no major change in the data transmitted from the linear acceleration sensor. To be safe, it is more appropriate to choose a hardware-based acceleration sensor. By the way, when I saw the pressure sensor, I thought that the sensor that monitors the pressure acting on the mobile phone was undoubtedly very suitable for identifying and knocking. I saw the description later and realized that it was the pressure monitoring.
As mentioned above, tapping operation on the mobile phone will lead to obvious changes in the data transmitted by the acceleration sensor. Therefore, in this function implementation, the method to determine whether there is a tapping operation is to detect whether the linear acceleration of the mobile phone has obvious changes compared to the normal situation. In order to eliminate the influence of gravity during the functional implementation, the data of the acceleration sensor needs to be processed to convert it into linear acceleration. Because conversion to linear acceleration is a process that requires calibration, a certain amount of data needs to be first invested for calibration to obtain more accurate linear acceleration. At the same time, considering that there may be scenes that may lead to misidentification in real life, such as shaking the mobile phone will bring the mobile phone a long and obvious linear acceleration change, so the concept of steady state is proposed, and the situation where the mobile phone is in relatively stable and there is no long-term obvious linear acceleration change is considered as steady state. Only in the case of steady state will the tapping bezel be recognized. In addition, this tapping recognition takes into account that the possibility of tapping on the mobile phone's border is too low, so only the tapping on the screen or back of the mobile phone can be considered. In this way, the data on the X and Y axis can be ignored during the recognition process, and only the linear acceleration of the Z axis can be considered.
2.2. Function introduction
The function implemented this time is to identify the tap operation on the mobile phone screen or back. The function implementation process: Register the sensor, collect data, and invest a specified number of data calibration to obtain a more accurate linear acceleration. After the calibration is completed, determine whether the current steady state is currently stable. If it is non-steady, wait for the next data. If it is steady state, call the method to determine whether there is a tap operation. While performing tap recognition, the linear acceleration and the number of recent taps will be processed. The steady state will be displayed on the interface.
2.3. Functional implementation
2.3.1. Obtain sensor data
The method of registering a sensor is a system-native method, so I will not explain it much. However, it should be noted that when registering an acceleration sensor, it is best to use SENSOR_DELAY_GAME to identify the sensor data sampling interval, because the acceleration data changes caused by knocking are very short. If SENSOR_DELAY_UI or SENSOR_DELAY_NORMAL are used, the acceleration changes caused by knocking are often not collected. Of course, if SENSOR_DELAY_FASTEST is used, there will naturally be no problem, but the performance consumption will be relatively large.
After registering the sensor, you can wait for data processing in the callback method. Below I will give the implementation code and comprehensively explain the implementation process.
public void onSensorChanged(SensorEvent sensorEvent) {
if ( == null) {
return;
}
if (() == accelerometerSensorType) {
float accelerationZ = [2];
if (accelerationZ > 0) {
recognitionKnockRatio = 20;
recognitionUniqueRatio = 10;
smoothSectionMaxRatio = 5f;
} else {
recognitionKnockRatio = 7.5f;
recognitionUniqueRatio = 6;
smoothSectionMaxRatio = 2.5f;
}
gravityZ = alpha * gravityZ + (1 - alpha) * accelerationZ;
linearAccelerationZ = accelerationZ - gravityZ;
if (calibrateLinearAcceleration) {
calibrateLinearAccelerationIndex++;
if (calibrateLinearAccelerationIndex <= calibrateLinearAccelerationSectionNumber) {
return;
}
calibrateLinearAcceleration = false;
}
if (sensorDataShowIndex >= sensorDataShowNumber) {
sensorDataShowIndex = sensorDataShowNumber - sensorDataShowDurationNumber;
Iterator<?> it = (0);
for (int i = 0; i < sensorDataShowDurationNumber; i++) {
();
();
}
(linearAccelerationZShowList);
}
(linearAccelerationZ);
sensorDataShowIndex++;
if (!stable) {
(linearAccelerationZ);
if (() >= stableSectionNumber) {
stableRecognition();
();
}
return;
}
knockRecognition(linearAccelerationZ);
}
}
The data obtained by the acceleration sensor are processed separately in the sensor data callback method. First, based on the positive and negative z-axis acceleration, different values are assigned to the three variables of recognitionKnockRatio, recognitionUniqueRatio, and smoothSectionMaxRatio. As for why this is done, it is because the actual tapping operation of Android phones is found that the acceleration sensor is sensitive to the feedback of the front tapping operation, and the feedback of the back tapping operation is relatively slow, and the feedback is feedback to the number. According to the level, the acceleration sensor data changes caused by tapping the front are much more obvious than the back, so different values must be assigned to the touch screen and the back. However, in fact, from the perspective of the mobile phone, it is impossible to analyze the obvious changes in acceleration caused by tapping operation originate from the touch front or the back. Therefore, the positive and negative of the z-axis acceleration is used to simply judge. After all, in most cases, the z-axis acceleration is positive, which means that the back of the mobile phone is biased towards the ground, and the user is more likely to tap the screen of the mobile phone, and the negative means that the screen of the mobile phone is biased towards the ground, and the user is more likely to tap the back of the mobile phone. As for the different sensitivity of the feedback of the acceleration sensor on the back, there are only two reasons for this. One is that the acceleration sensor is closer to the screen than the back, and the other is the problem with the case of the Android phone. This is particularly obvious on the LG G3. The LG G3 has a plastic shell with a certain arc, and the change in sensor data caused by tapping on the back is much lower than that of tapping on the back. The Samsung S6 with a metal shell has a sensor data change in sensor data caused by tapping on the back is close to that of tapping on the back. In fact, the above three coefficients are empirical coefficients, and different values should be provided as much as possible for different types of mobile phones. For reasons, please refer to the LG G3 and Samsung S6 mentioned earlier. Once again, I feel the diversity of Android phones. There are too many types of Android phones and the different hardware designs make it impossible to apply on one phone to another. If there are only those models like the iPhone, it will undoubtedly be much easier to deal with.
Then filter the acceleration to obtain linear acceleration. The method of obtaining linear acceleration refers to the method suggested in the source code of Android SensorEvent:
* <p>
* It should be apparent that in order to measure the real acceleration of
* the device, the contribution of the force of gravity must be eliminated.
* This can be achieved by applying a <i>high-pass</i> filter. Conversely, a
* <i>low-pass</i> filter can be used to isolate the force of gravity.
* </p>
*
* <pre class="prettyprint">
*
* public void onSensorChanged(SensorEvent event)
* {
* // alpha is calculated as t / (t + dT)
* // with t, the low-pass filter's time-constant
* // and dT, the event delivery rate
*
* final float alpha = 0.8;
*
* gravity[0] = alpha * gravity[0] + (1 - alpha) * [0];
* gravity[1] = alpha * gravity[1] + (1 - alpha) * [1];
* gravity[2] = alpha * gravity[2] + (1 - alpha) * [2];
*
* linear_acceleration[0] = [0] - gravity[0];
* linear_acceleration[1] = [1] - gravity[1];
* linear_acceleration[2] = [2] - gravity[2];
* }
* </pre>
The acceleration is processed through high-pass filtering and low-pass filtering to eliminate the influence of gravity to obtain linear acceleration, but in this process a certain amount of data needs to be passed in for calibration to obtain more accurate linear acceleration. Here we set the calibrateLinearAccelerationSectionNumber as the data length for calibration data, and use calibrateLinearAccelerationIndex and calibrateLinearAcceleration to control when calibration is over.
After calibration, use linearAccelerationZShowList to store the linear acceleration of the sensor displayed on the application interface. Then, if it is in a non-steady state, steady-state recognition will be started, and whether the current state is steady-state. If it is in a steady-state state, tap recognition will be started.
2.3.2. Steady-state recognition
As mentioned above, if a user performs operations such as shaking the phone, it will produce obvious acceleration changes, which may lead to misidentification. Therefore, the concept of steady state is proposed here, that is, the state where the acceleration of the phone does not change significantly for a long time. Extending to the real scene, the state where the user does not move the phone significantly. Strictly speaking, the possibility of a general user tapping while shaking the phone is extremely low, so the concept of steady state can be officially applied to the functional implementation.
I have already understood the concept of steady state, so how should we define what situations belong to steady state and what situations belong to non-steady state? I will give the implementation code and explain it in a comprehensive way.
private void stableRecognition() {
int exceptionNumber = 0;
float accelerationZValue;
float minAccelerationZValue = Integer.MAX_VALUE;
float maxAccelerationZValue = Integer.MIN_VALUE;
for (int i = stableSectionNumber - 1; i >= 0; i--) {
accelerationZValue = (i);
if ((accelerationZValue) > maxStableOffset) {
exceptionNumber++;
} else {
if (accelerationZValue > maxAccelerationZValue) {
maxAccelerationZValue = accelerationZValue;
} else {
if (accelerationZValue < minAccelerationZValue) {
minAccelerationZValue = accelerationZValue;
}
}
}
}
stable = exceptionNumber <= maxExceptionNumber;
if (stable) {
if (linearAccelerationZStableSection == 0) {
linearAccelerationZStableSection =
(maxAccelerationZValue - minAccelerationZValue) / 2;
}
if (linearAccelerationZStableSection > maxStableOffset) {
linearAccelerationZStableSection = maxStableOffset;
}
}
(stable);
("stable", "" + stable);
("exceptionNumber", "" + exceptionNumber);
("linearAccelerationZStableSection", "" + linearAccelerationZStableSection);
}
During this function implementation, the way to judge steady state is to sample 50 points and then calculate the absolute value of each point. If it is greater than the maximum deviation maxStableOffset, it is regarded as an exception point, and if the exception point is greater than the maximum number of abnormal points, it is regarded as non-stable state, and vice versa. After judging that the steady state is over, if it is in steady state, half of the difference between the maximum acceleration and the minimum acceleration of the Z-axis after the abnormal point data is removed is considered as the fluctuation interval linearAccelerationZStableSection. maxStableOffset and maxExceptionNumber are both empirical coefficients, and are derived from linear acceleration analysis in different scenarios actually provided by Android phones. There is a problem now, that is, if the original state is in steady state, then the user suddenly operates the phone and changes the phone state to non-steady state, then what should be done? Don't worry, this problem will be handled during the tap recognition process.
2.3.3. Hit recognition
Now we have reached the core of the entire function implementation: tap recognition. As mentioned above, tap will cause obvious changes in acceleration sensor data, but how do we use code to detect taps, and how to eliminate misidentification problems caused by users to other mobile phone operations? In fact, these problems will be handled here. Now I will give the implementation code and comprehensive code to explain it.
private void knockRecognition(float linearAccelerationZ) {
float linearAccelerationZAbsolute = (linearAccelerationZ);
float linearAccelerationZAbsoluteRadio =
linearAccelerationZAbsolute / linearAccelerationZStableSection;
if (linearAccelerationZAbsoluteRadio > recognitionUniqueRatio) {
(linearAccelerationZ);
currentForecastNumber = forecastNumber;
} else {
if (() > 0) {
if (currentForecastNumber > 0) {
currentForecastNumber--;
} else {
handleUniqueLinearAccelerationZ();
}
}
}
if (linearAccelerationZAbsoluteRadio < smoothSectionMaxRatio) {
float offsetWeight = 0.001f;
linearAccelerationZStableSection =
weightedMean(offsetWeight, linearAccelerationZAbsolute,
linearAccelerationZStableSection);
}
}
knockRecognition is a method used to process linear acceleration and then confirm whether there is a knock operation. First, process the linear acceleration in the incoming parameter to obtain the absolute value of linear acceleration. Then, if the ratio of the absolute value of linear acceleration to the fluctuation interval is greater than the recognitionUniqueRatio, then it is believed that the phone is being affected by force. In order to determine whether it is a knock operation or other operation of the user, first add the linear acceleration to the unique linear acceleration list. On the contrary, if it is less than or equal to recognitionUniqueRatio, it is considered that the phone is in a relatively stable state. At this time, if the length of the unique linear acceleration list is greater than 0, if the currentForecastNumber is greater than 0, the currentForecastNumber is less than or equal to 0, the unique linear acceleration list is started to be processed, and in the process of processing the unique linear acceleration list, it officially begins to recognize whether to hit and whether the current state changes to non-stable state. While performing the above operation, if the ratio of the absolute value of linear acceleration to the fluctuation range is less than smoothSectionMaxRatio, the absolute value of linear acceleration is used to smooth the fluctuation range.
Here, you must have questions about currentForecastNumber, what does this variable mean and why this variable is there? This is why a single knock may cause two unique linear accelerations that are close but discontinuous. Without the currentForecastNumber variable, it will cause a real-life one hit that may be recognized as two hit operations.
If the ratio of the absolute value of linear acceleration to the fluctuation range is less than smoothOffsetMaxRatio, the absolute value of linear acceleration is used to smooth the fluctuation range. This is because on the one hand, the state of the mobile phone may change at any time, and the fluctuation range should change with the change of the mobile phone state. On the other hand, there may be problems with the fluctuation range calculated during steady-state recognition and cannot correctly reflect the current acceleration fluctuation of the mobile phone. At this time, you need to learn based on the latest data to smooth the fluctuation range. Why is the ratio smaller than smoothSectionMaxRatio? The linear acceleration greater than smoothSectionMaxRatio is basically abnormally illiquid range, and is not suitable for smoothing fluctuation intervals. If the ratio of linear acceleration to fluctuation intervals in reality basically exceeds smoothSectionMaxRatio, it means that the mobile phone is mostly in a non-steady state now, and you can just wait for the new steady-state recognition to reset the fluctuation interval. In addition, as mentioned above, recognitionUniqueRatio and smoothOffsetMaxRatio are empirical coefficients and can be set independently.
private void handleUniqueLinearAccelerationZ() {
("linearAccelerationZStableSection", "" + linearAccelerationZStableSection);
int recognitionKnockNumber = 1;
int uniqueLinearAccelerationZListLength = ();
float accelerationZOffsetAbsolute;
float maxAccelerationZOffsetAbsolute = 0;
for (int i = 0; i < uniqueLinearAccelerationZListLength; i++) {
accelerationZOffsetAbsolute = ((i));
if (maxAccelerationZOffsetAbsolute < accelerationZOffsetAbsolute) {
maxAccelerationZOffsetAbsolute = accelerationZOffsetAbsolute;
}
("uniqueLinearAccelerationZList index" + i,
"" + (i));
}
();
("uniqueLinearAccelerationZListLength",
"" + uniqueLinearAccelerationZListLength);
if (uniqueLinearAccelerationZListLength > unstableListLength) {
stable = false;
(stable);
return;
}
("maxAccelerationZOffsetAbsolute / linearAccelerationZStableSection",
"" + (maxAccelerationZOffsetAbsolute / linearAccelerationZStableSection));
if (maxAccelerationZOffsetAbsolute >
linearAccelerationZStableSection * recognitionKnockRatio) {
("recognitionKnockRatio", "" + recognitionKnockRatio);
("recognitionUniqueRatio", "" + recognitionUniqueRatio);
knockRecognitionSuccess(recognitionKnockNumber);
}
}
Finally, we have reached the final handleUniqueLinearAccelerationZ method, as the name suggests, is used to process the unique linear acceleration list. In this method, tap recognition and determination are made on whether the steady state is changed. If the length of the unique linear acceleration list exceeds the length of the non-stable unique linear acceleration list, it is believed that the current state of the mobile phone is now changed to non-stable state and ends the method. If it is found that the maximum offset value in the acceleration offset data list exceeds a certain multiple of the fluctuation range, it is recognized as a knock.
3. Summary
At this point, we have finished the process of tapping and recognition. In fact, the knock recognition method I provide still has misidentification. I have used ios Knock, which has the ability to meet the price, and the recognition rate is quite good. I don’t know whether they use machine learning or other methods to attribute their recognition coefficients. Of course, the knock recognition I provide here is just a knock recognition method, and I can’t say it is mature, because it has not been tested by real users. Everyone can completely change the algorithm or even replace the sensor according to their own ideas to achieve their own knock recognition. And I am actually equivalent to providing an implementation idea here.
This is the third blog. The first blog is a relatively biased but not easy to deal with. I chose a small module that I have done in the test. I wrote the ID3 tag to the MP3 file. The second blog chose a very rigorous practical module: audio synthesis. The first two modules have one thing in common, that the various specifications are already very clear. Although the code implementation may be different, the implementation ideas must be the same. The third blog's tap detection is undoubtedly much looser. So I wrote the section on the implementation ideas for the first time, because I am not sure whether my implementation ideas are completely correct. As a practical application of sensors, there are countless possibilities. We can try it according to our own ideas. If we are wrong, we can just change the direction.
The above is the Android phone screen tapping unlock implementation code introduced to you by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support for my website!