Introduction
Use redis to achieve accurate search of all articles to paragraphs
Implementation ideas
Article segmentation, use a form to record paragraph information separately
Participle of paragraphs, divide paragraphs into words
After word segmentation, record it in redis using an ordered set, and after each word, record the paragraph ID set containing the word segmentation.
Use a hash key to record the word segmentation of each paragraph, which is used to delete word segmentation information
When searching, first divide the word, and then search the corresponding paragraphs of the word segment according to the divided word.
Return result
Specific implementation
Article segmentation
Article paragraph table structure
type TextModel struct {
ArticleID uint `json:"articleID"`
Head string `json:"head"`
Body string `json:"body"`
}
Segmentation functions
// Because there is a loop guide package in my project, it is not used to process it with Gorm's mapping table.type TextModel struct {
ArticleID uint `json:"article_id"`
Head string `json:"head"`
Body string `json:"body"`
}
func MdContentTransformation(id uint, title string, content string) (list []TextModel) {
lines := (content, "\n")
var headList []string
var bodyList []string
var body string
headList = append(headList, title)
var flag bool
for _, line := range lines {
if (line, "```") {
flag = !flag
}
if !flag && (line, "#") {
// Title line headList = append(headList, getHead(line))
if (body) != "" {
bodyList = append(bodyList, getBody(body))
}
body = ""
continue
}
body += line
}
if body != "" {
bodyList = append(bodyList, getBody(body))
}
if len(headList) > len(bodyList) {
bodyList = append(bodyList, "")
}
if len(headList) != len(bodyList) {
("headList and bodyList are inconsistent \n headList:%q %d\\n bodyList: %q %d\n", headList, len(headList), bodyList, len(bodyList))
return
}
for i := 0; i < len(headList); i++ {
list = append(list, TextModel{
ArticleID: id,
Head: headList[i],
Body: bodyList[i],
})
}
return
}
func getHead(head string) string {
s := (((head, " ")[1:], " "))
return s
}
func getBody(body string) string {
body = (body)
return body
}
Participle words
Use third-party libraries for word segmentation
/go-ego/gse
Third-party library download
go get -u /go-ego/gse
Library initialization
func InitGse() {
newGse, _ := ()
global_gse.Gse = newGse
}
Word participle function
func textParticiple(textList ...string) (words []string) {
for _, text := range textList {
word := global_gse.(text, true)
words = append(words, word...)
}
return
}
Save the divided words into redis
Use redis to collect the paragraph IDs corresponding to each word
func SetTextSearchIndex(textID uint, words []string) {
for _, word := range words {
if word == "" {
continue
}
(("text_%s", word), textID)
}
}
Use the hash key to record the corresponding paragraphs and word information of each article, and use it to delete information
func SetTextSearchWords(articleID uint, textID uint, words []string) {
_words, _ := (words)
(("text_search_words_%d", articleID), (int(textID)), _words)
}
Query operation
Get a collection of paragraphs by querying words after text participle
func GetTextSearchIndex(text string) []string {
//Part participle words := global_gse.(text, true)
var _words []string
for _, word := range words {
_words = append(_words, ("text_%s", word))
}
vals, _ := (_words...).Result()
return vals
}
Query the database based on the id list found
idList := redis_article.GetTextSearchIndex()
query := ("id in ?", idList)
Subsequent processing to make the searched text highlighted
You can put the searched text in a specific tag and then return the information
The above is the detailed explanation of golang's full-text search function using redis. For more information about goredis full-text search, please follow my other related articles!