SoFunction
Updated on 2025-03-10

Use shell to read ini file method steps

motivation

I decided to write a script for clean initial setup of macOS (BSD)/Linux. I think there is a tsukkomi that says "there are now the perl/python preinstalled" but I decided to use Shell scripts (bash) because it's easy to write process processes. However, writing various configuration files in shell syntax is unreadable, so I decided to write the configuration file into a .ini file, process it with sed, and then load it into the shell. Regarding parsing .ini files with sed, various examples appear if you search on Google, but I decided to use it as a specification that can handle .ini files (such as) and be more readable by referencing them.

.ini file format

  • Set the section in [section]. A sequence of whitespace characters immediately after [and immediately before] (space/tab) is not considered part of a partial name. (It is stipulated that space characters can be placed between the section name and its parentheses before and after.) However, section names should not contain line breaks.
  • Use parameter = value to set parameter variables and their values. = There can be spaces before and after. Assume that the parameter name does not include =. If the parameter name contains characters that cannot be used in the shell variable, replace these characters with _ in the output shell variable name.
  • If there is \ at the end of the line, the next line is considered a continuation line. If a line has 4 or more spaces/tabs at the beginning, it is considered a continuation of the previous line.
  • Ignore the end of the line in #,; as a comment.
  • Ignore rows that do not contain [and’]’, or rows that do not contain = at positions outside the beginning of the line.

What to do when processing .ini files in shell scripts

View a partial list.

Read the argument as a shell variable. However, as a variant

1. Make the parameters of a specific part a shell variable. In some cases, set it as a local variable or an environment variable in the shell function.
2. Make all or part of the parameters as shell variables. The shell variable name is 'prefix based on the segment name'+'parameter name'. Also in this case, set it as a local variable or an environment variable. A prefix based on a section name is generated by replacing characters that cannot be used in the shell variable in the section name string with _ and adding _ at the end.

Assuming processing system

  • bash
  • BSD sed (I can't use the convenient GNU sed extension, but I want to run it on a clean macOS.) However, I use an extension regex with the -E option. (Because if one or more matching metacharacters '+' cannot be used, the sed statement will become longer.)

Handling continuous lines and comments seems to have a more general purpose, so I describe it separately. This is achieved by adding the code to what is described there.

View some list

This is relatively easy. Do not deal with lines that do not match the section name format. All you have to do is delete the '[',']' and unnecessary parts of the line corresponding to the section name format and output. At this time, it should be noted that line breaks are not deleted casually. As described separately, expressions other than line breaks ([^ \ [:space:]]] | [[ : blank: ]]) are used.

:begin
$!N
s/[#;]([^[:space:]]|[[:blank:]])*([^\\[:space:]]|[[:blank:]])(\n)/\3/
s/[#;]([^[:space:]]|[[:blank:]])*(\\)(\n)/\2\3/
$s/[#;]([^[:space:]]|[[:blank:]])*$//
/(\\\n|\n[[:blank:]]{4})/ {
  s/[[:blank:]]*(\\\n|\n[[:blank:]]{4})[[:blank:]]*/ /
  t begin
}
/^[[:blank:]]*\n/ D
/\n[[:blank:]]*$/ {
  s/\n[[:blank:]]*$//
  t begin
}
/^\[([^[:space:]]|[[:blank:]])*\]/! D
s/\[[[:blank:]]*//
s/[[:blank:]]*\]([^[:space:]]|[[:blank:]])*//
P
D

For example, if you try to use it as an example .ini file,

# -*- mode: conf-mode ; -*- \
#;
#; sample .ini file
#;
#;

[tako]
param_a=(1 # 2 3 \
4 5 ### 6 7 
    8 9 # 10
    )

a=b # kani \
# kani \


[kani]
param_a=1
param_b=2

[uni]
param_a=3
param_b=4

[wani]
param_a=5
param_b=6

[hebi]
param_a=9
param_b=10

output example:

% sed -nE -f list_section.sed 
tako
kani
uni
wani
hebi

Extract only specific parts

The following example extracts only rows of a specific part (wani) from the above example, which follow the correct parameter definition format. If you find a line that exists in the form of a section name, store the section name in the reserved space, otherwise if the contents of the reserved space do not match the required section name, go to the next line. ...If it matches, check whether the definition of the parameter format is matched, remove spaces, add text in this example to make it a local variable of the shell function, and you can summarize the shell variable definition. At the end of the line. Because it has become longer, comment lines are added, but as a control structure, each processing is necessary. If the subsequent processing is not necessary, it will go back to the beginning. , which should be easier to understand.

:begin
$!N
# Erase comment strings
s/[#;]([^[:space:]]|[[:blank:]])*([^\\[:space:]]|[[:blank:]])(\n)/\3/
s/[#;]([^[:space:]]|[[:blank:]])*(\\)(\n)/\2\3/
$s/[#;]([^[:space:]]|[[:blank:]])*$//
# Concatenate continuation lines
/(\\\n|\n[[:blank:]]{4})/ {
  s/[[:blank:]]*(\\\n|\n[[:blank:]]{4})[[:blank:]]*/ /
  t begin
}
# Erase blank lines
/^[[:blank:]]*\n/ D
/\n[[:blank:]]*$/ {
  s/\n[[:blank:]]*$//
  t begin
}
# Check section headline and store section name to holdspace
/^([^[:space:]]|[[:blank:]])*\[([^[:space:]]|[[:blank:]])*\]/ {
h
x
s/^([^[:space:]]|[[:blank:]])*\[(([^[:space:]]|[[:blank:]])*)\].*$/\2/g
s/^[[:blank:]]*//g
s/[[:blank:]]$//g
x
D
}
# Skip line if current section is not interested one
x
/^wani$/! { 
  x
  D
}
x
# Print if it is proper parameter definition 
/^(([^[:space:]=]|[[:blank:]])*)=(([^[:space:]]|[[:blank:]])*)/ {
  s/^[[:blank:]]*/local /
  s/[[:blank:]]*=[[:blank:]]*/=/
  s/(([^[:space:]]|[[:blank:]])*)[[:blank:]]*(\n)/\1;\3/
  P
}
D

How to qualify shell variable names

As stated in the section on what you want to do, you want to add a prefix generated from the partial name to the shell variable, or if the parameter name in the .ini file contains a string that cannot be used in the shell variable, convert appropriately. As a simple method, there are methods to call sed multiple times and process, but if you can, I think it is more beautiful to handle it with a process number sed. The biggest limitation here is that sed does not have a save space. In a high-level scripting language, text segments can be divided into multiple variables for storage, processing, and combining. On the other hand, in the case of sed, the standard approach seems to be to save and use multiple text as a stack and use reserved space as a separator with line breaks.

For example, in the following example, the reserved space starts from the first row.

  • Partial name
  • Prefixes with formatted part names
  • Backup of the first line of the pattern space
  • Backup of the second line of the pattern space
  • Shell variable name formatted based on parameter names in .ini file

You need to decide how to use it, keep the number of rows in the space unchanged (2 rows in the following example), and restore the mode space appropriately when processing row conversions. Since the commands for switching keep space and pattern space are only gG and hH, similar processing may occur repeatedly, so it is undeniable that it is really beautiful to do in the 1sed process.

Anyway, here is an example of extracting the wani and uni parts from the above file and outputting the definition statement for shell variables with partial names added. The overall control structure is kept simple and I added comments so I hope you can see where the rewrite is to extract another section, for example.

# Initialine the hold space: (Single empty line at the beginning)
1 { H ; x ; 
  # Change the expression for the defalut section name and/or variable prefix, here.
  s/^([^[:space:]]|[[:blank:]])*(\n)([^[:space:]]|[[:blank:]])*$/global\2global_/g
  x
}
:begin
$!N
# Erase comment strings
s/[#;]([^[:space:]]|[[:blank:]])*([^\\[:space:]]|[[:blank:]])(\n)/\3/
s/[#;]([^[:space:]]|[[:blank:]])*(\\)(\n)/\2\3/
$s/[#;]([^[:space:]]|[[:blank:]])*$//
# Concatenate continuation lines
/(\\\n|\n[[:blank:]]{4})/ {
  s/[[:blank:]]*(\\\n|\n[[:blank:]]{4})[[:blank:]]*/ / ; t begin
}
# Erase blank lines
/^[[:blank:]]*\n/ D
/\n[[:blank:]]*$/ {
  s/\n[[:blank:]]*$// ; t begin
}
# Check section headline and store section name to holdspace
/^([^[:space:]]|[[:blank:]])*\[([^[:space:]]|[[:blank:]])*\]/ {
  # Remove blackets and extra spaces at first line
  s/^([^[:space:]]|[[:blank:]])*\[(([^[:space:]]|[[:blank:]])*)\](([^[:space:]]|[[:blank:]])*)(\n)/\2\6/g
  s/^[[:blank:]]*//g; s/[[:blank:]]*(\n)/\1/g
  # Store the section name to the hold space, and format stored one for shell variable for the hold space
  h
  x
  s/(\n)([^[:space:]]|[[:blank:]])*$//
  s/([^[:alnum:]_]|$)/_/g
  x
  # Append the section name to the hold space.
  H
  # Remove unused line of the hold space and rearrange the remaining lines.
  x
  s/(([^[:space:]]|[[:blank:]])*)(\n)(([^[:space:]]|[[:blank:]])*)(\n)(([^[:space:]]|[[:blank:]])*)$/\4\3\1/
  x
  D
}
# Skip line if current section is not interested one
x
/^(wani|uni)(\n)/! { x ; D ; }
x
# Print if it is proper parameter definition 
/^(([^[:space:]=]|[[:blank:]])*)=(([^[:space:]]|[[:blank:]])*)/ {
  # Store current patern space text at the end of the hold space
  H

  # Build shell script variable name and store it at the end of the hold space
  s/(([^[:space:]=]|[[:blank:]])*)=.*$/\1/g
  s/^[[:blank:]]*//
  s/[[:blank:]]*$//
  s/[^[:alnum:]_]/_/g
  # If further rename of the variable name is necessary, put here.

  # Store variable name at the end of the hold space
  H

  # Build parameter variable value and keep it at pattern space
  # At first, Resore the current line text from hold space
  g
  # Remove unused lines.
  s/^(([^[:space:]]|[[:blank:]])*\n){2}//
  s/(\n([^[:space:]]|[[:blank:]])*){2}$//
  # Remove the text other than the parameter value
  s/^([^[:space:]=]|[[:blank:]])*=//g
  # If further formatting of the value is necessary, put here.

  # Append hold space stored date into pattern space
  G
  # Remove unused lines from the pattern space
  s/^(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*\n)/\1\5\9/
  # Rearrance the order of line in the pattern space, it is nessacery because only \1 ...\9 is avaiable
  s/^(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*\n)(([^[:space:]]|[[:blank:]])*)(\n)(([^[:space:]]|[[:blank:]])*)$/\1\3\8\7\5/

  # Format the output in the first line of the pattern space, and 
  # Restore the next line at the second line of the pattern space
  s/^(([^[:space:]]|[[:blank:]])*)(\n)(([^[:space:]]|[[:blank:]])*)(\n)(([^[:space:]]|[[:blank:]])*)(\n([^[:space:]]|[[:blank:]])*)$/local \4\7=\1;\9/

  # Clean up hold space
  x
  s/(\n([^[:space:]]|[[:blank:]])*){3}$//
  x
  P
}
D

output example:

% sed -n -E -f pickup_section2.sed
local uni_param_a=3;
local uni_param_b=4;
local wani_param_a=5;
local wani_param_b=6;

Shell scripting

It is very troublesome to rewrite the sed file every time I change the section to be extracted or switch the output format (sh/csh, shell variables/local variables/environment variables, ON/OFF of variable name prefix), so I prepared a shell script to generate the sed command as an option on the command line. Generate using the template from another article. For the above seds, I also tried selecting the output by the parameter variable name (shell variable name) of the .ini file. (However, if multiple segment names and variable names are specified, the combination is not unique, so it may not be a useful feature.)

File storage:

/nanigashi-uji/parse_ini_sh
/nanigashi_uji/parse_ini_sh

How to use

[Usage] % parse_ini.sh -list     file [files ...]
        % parse_ini.sh [options] file [files ...]

[Options]
    -l,--list                       : List sections 
    -S,--sec-select       name      : Section name to select
    -T,--sec-select-regex expr      : Section reg. expr. to select
    -V,--variable-select name       : variable name to select
    -W,--variable-select-regex expr : variable reg. expr. to select
    -L,--local                      : Definition as local variables (B-sh)
    -e,--env                        : Definition as enviromnental variables
    -q,--quot                       : Definition by quoting with double/single-quotation.
    -c,--csh,--tcsh                 : Output for csh statement (default: B-sh)
    -b,--bsh,--bash                 : Output for csh statement (default)
    -s,--sec-prefix                 : add prefix: 'sectionname_' to variable names. 
    -v,--verbose                    : Verbose messages 
    -h,--help                       : Show Help (this message)

output example:

% parse_ini.sh --list
tako
kani
uni
wani
hebi

% parse_ini.sh -S kani -L
local param_a=1;
local param_b=2;

% parse_ini.sh -S kani -L -s
local kani_param_a=1;
local kani_param_b=2;

% parse_ini.sh -S kani -L -e -c
setenv param_a 1;
setenv param_b 2;

% parse_ini.sh -S kani -L -e -c -q
setenv param_a "1";
setenv param_b "2";

This is the article about the steps of using shell to read ini files. For more relevant shells to read ini files, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!