This is an old revision of the document!
Metadata Management via CLI
The iCommands
allow to execute most metadata tasks also via command-line interface (CLI). This section will explain how to:
- Read/List metadata
- Create metadata
- Modify metadata
- Set new metadata value
- Remove metadata
- Copy metadata
- Query metadata
While the iCommands Wiki section contains the information to get you started using the iCommands
and also lists the commands that can be used for metadata management, the current section will further built on top of this by providing more detailed examples.
Notes:
- If you want to work with metadata templates, you have to use the web portal for the creation and filling of the templates.
- Within the RDMS, metadata is always added as so called 'attribute-value-units' triples, AVUs. The 'attribute' specifies the metadata key (e.g. 'Author') while the 'value' specifies the corresponding value for this key (e.g. 'John Doe'). The 'unit' entry is optional, but can be useful in cases where the entries could be otherwise ambiguous (e.g. A: 'Distance', V: '100', U: 'km'). This can be left blank if no unit is desired.
- When working with the
imeta
command, you have to specify if you are working on a file (data object) or folder (collection). This can be achieved by specifying the additional-d
(for files) or-C
(for folders) flags when using the command (see below for examples). - For all of the
imeta
command, you can use single or double quotes around an entry if it includes blanks, for exampleimeta add -d <path> Author 'Name Surname
'.
iCommands for Metadata Management
The imeta
command is the relevant sub-command from the iCommands
packages that is used to work with metadata from the CLI.
You can execute the following command from the CLI to get an explanation of the imeta
command as well as an overview of its possible usage:
$ imeta help
Furthermore, you can execute imeta help <subcommand>
to get a more detailed help displayed for the respective subcommand of imeta
.
For example imeta help add
would display information about the imeta add
subcommand which can be used to add metadata from the CLI (see below for examples).
Read/List Metadata
Adding Metadata
Modifying Metadata
Set Metadata
Removing Metadata
Copy Metadata
To copy metadata to a file/folder from another, the following commands is used:
$ imeta cp [-d, -C] <path_soruce> <path_dest>
where <path_source>
specifies the relative or absolute path to the data from which the metadata should be copied, and<path_source>
specifies the relative or absolute path to the data to which the metadata should be copied.
Note:
This functionality is currently blocked on our system!
Query Metadata
To query for metadata, the following command is used:
$ imeta qu [-d, -C] <Attribute> <Operator> <Value> [...]
where <Attribute>
and Value
are again the AVUs for which the query should be performed, and <Operator>
is the respective relation between those.
Possible operators are:
<
: Used to query for metadata where the value associated with an attribute is smaller than a certain value.>
: Used to query for metadata where the value associated with an attribute is bigger than a certain value.<=
: Used to query for metadata where the value associated with an attribute is smaller or equal than a certain value.>=
: Used to query for metadata where the value associated with an attribute is bigger or equal than a certain value.=
: Used to query for metadata where the value associated with an attribute is equal to a certain value.!=
: Used to query for metadata where the value associated with an attribute is not equal to a certain value.<>
: Same as!=
like
: Used to query for metadata where the value associated with an attribute is like a certain value and which allows using wildcards for the value.not like
: Used to query for metadata where the value associated with an attribute is not like a certain value and which allows using wildcards for the value.
Examples:
In the presented case, we could use the imeta qu
command to query for all folders or files where the Author: Tester_2
using the =
operator:
$ imeta qu -d Author = Tester_2 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9
We could also use the like
operator to extend our query not just to the cases where Author: Tester_2
, but also Author: Tester_1
by using using wildcards at the correct places:
# A general query with '%' wildcard. Note that for example_file_8, we defined 'Author: Tester_1' while for example_file_9 we defined 'Author: Tester_2'. Both are still found. $ imeta qu -d Author 'like' Tester% collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_8 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9 # Difference between '%' and '_' wildcards using the following two example files. $ imeta ls -d example_file_7 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_7: attribute: Author value: First Second units: $ imeta ls -d example_file_6 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_6: attribute: Author value: First Second units: # Trial using the '%' which replace a whole string or multiple strings. $ imeta qu -d Author 'like' 'First %' collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_6 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_7 # In contrast, '_' does just replace a single digit. $ imeta qu -d Author 'like' 'First _' No rows found $ imeta qu -d Author 'like' 'First Secon_' collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_6 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_7 $ imeta qu -d Author 'like' 'First S_cond' collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_6 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_7
Notes about using wildcards:
- Two wildcards can be used in the queries:
%
and_
. - While
%
will replace any term,_
will just act as wildcard for one letter (see the second query example above that shows the difference). - The placeholder can be used not just at the beginning/end of the term for which is queried, but instead at any place.
It is furthermore possible to use numerical comparison operators like >
, <
or >=
<=
to just query results where a value is 'bigger', 'smaller', 'bigger or equal' or 'smaller or equal' to the specified query value.
# Show results which have 'Distance: 140' as metadata. $ imeta qu -d Distance = 140 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_8 # Show results which have 'Distance: 150' as metadata. $ imeta qu -d Distance = 150 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9 # Show results which have a metadata key with an associated value of less than 150. $ imeta qu -d Distance '<' 150 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_5 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_8 # Show results which have a metadata key with an associated value of more or equal to 150. $ imeta qu -d Distance '>=' 150 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9 # Show results which have a metadata key with an associated value of more as 150. $ imeta qu -d Distance '>' 150 No rows found
Notes about using numerical comparison:
- The numerical comparison only works for decimal notation (e.g. 10 > 9 or 2.1 < 2.4), but it does not work in cases of other notations (e.g. 1e3 > 1e2 or 2e-4 < 1e-3).
Also, queries can be used to query/filter for multiple, combined metadata entries
# First see which metadata is added to the files. $ imeta ls -d example_file_8 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_8: attribute: Author value: Tester_1 units: ---- attribute: Distance value: 140 units: km $ imeta ls -d example_file_9 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_9: attribute: Author value: Tester_2 units: ---- attribute: Distance value: 150 units: km # Use a single and combined query to see the difference in the results. $ imeta qu -d Distance '>=' 140 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_8 ---- collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9 $ imeta qu -d Distance '>=' 140 and Author = Tester_2 collection: /rug/home/rdms-testers@rug.nl dataObj: example_file_9
Using the Interactive Mode
It is also possible to use the imeta
command in an interactive mode. To enter the interactive mode, just type imeta
without any additional subcommand.
You can see that you are in the interactive mode of imeta
when the imeta>
is displayed in your terminal.
Once in the interactive mode, subsequent commands can be added after each other. If you want to leave the interactive mode again, type q
or quit
and confirm.
Examples:
# First check that the metadata that is currently set for the example file. $ imeta ls -d example_file_5 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_5: None # Enter the interactive mod and subsequently execute different imeta tasks before exiting the interactive mode. $imeta imeta>add -d example_file_5 Type 'Test File' imeta>add -d example_file_5 Author "John Doe" imeta>add -d example_file_5 Distance 10e-5 km imeta>quit # Check again to see if all tasks were executed. $ $ imeta ls -d example_file_5 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_5: attribute: Author value: John Doe units: ---- attribute: Distance value: 10e-5 units: km ---- attribute: Type value: Test File units:
Further Tips
This section contains some further important and helpful tips for the working with metadata from the CLI.
Copying Files and Metadata
When copying a folder/file that contains metadata, the metadata is not copied to the copy of the file/folder!
# Check metadata for the original $ imeta ls -d example_file_9 AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_9: attribute: Author value: Tester_2 units: ---- attribute: Distance value: 150 units: km # Make a copy of the file. $ icp example_file_9 example_file_9_copy # Check metadata for the copy. $ imeta ls -d example_file_9_copy AVUs defined for dataObj /rug/home/rdms-testers@rug.nl/example_file_9_copy: None
Advanced Usages
Of course, as with other iCommands
, it is possible to include the imeta
sub-command in scripts etc. to execute more advanced/extended tasks.
The interactive imeta
mode can be also used to pipe information into it and then automatically execute the desired tasks.
For example, one could combine and iquest
query to first select files that fulfill the query condition and then use this information to set metadata on all those files with one terminal statement:
# This example will first query for all files in the collection at '<path>' that have a name of the type 'TEST%' (e.g. 'TEST_Sample', 'TEST_Reaction', etc.). # It will then use this query result and use them for the 'set -d <path>/<filename> <attribute> <value> <unit>' statement. # This will then be piped into 'imeta'. As a result the AVUs specified in the 'set' statement will be added to all files that met the query condition. iquest 'set -d %s/%s <attribute> <value> <unit>' "select COLL_NAME, DATA_NAME where COLL_NAME = '<path>' and DATA_NAME like 'TEST%'" | imeta