Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
rdms:webapp:search [2024/04/19 09:58] – [Sub-Searches] burcurdms:webapp:search [2024/11/15 12:29] (current) burcu
Line 2: Line 2:
 ====== Search ====== ====== Search ======
  
-The RDMS web interface comes with and extensive and flexible search functionality. Unlike other storage solutions, this search engine offers the advantage of utilizing metadata for exact data queries. Additionally, the integrated search engine also allows to make more elaborated queries/searches, such as combining multiple metadata entries.+The RDMS web interface comes with an extensive and flexible search functionality. Unlike other storage solutions, this search engine offers the advantage of utilizing metadata for exact data queries. Additionally, the integrated search engine also allows you to make more elaborate queries/searches, such as combining multiple metadata entries.
  
-This section will explain the basic functions of the integrated search and provide some more elaborated search/query examples using metadata. +This section will explain the basic functions of the integrated search and provide some more elaborate search/query examples using metadata. 
  
 ===== Accessing RDMS Search ===== ===== Accessing RDMS Search =====
  
-Accessing the search engine is possible via the left-sided menu in the web interface. +You can access the search engine via the left-sided menu in the web interface. 
  
-After logging in to the interface, simply click the {{:rdms:webapp:search.png?nolink&30|}} button on the left side of the menu to open the search function in the current window.+After logging in to the interface, click the {{:rdms:webapp:search.png?nolink&30|}} button on the left side of the menu to open the search function in the current window.
  
 {{ :rdms:webapp:rdms_search_general_1.png?direct&800 |}} {{ :rdms:webapp:rdms_search_general_1.png?direct&800 |}}
Line 21: Line 21:
   - **Key/Value/Extra (KVE) search:** These search options is specifically designed for querying using multiple metadata entries simultaneously. You can access these search options via the {{:rdms:webapp:wiki_search_avu_1.png?nolink&20|}} button at the top of the search interface.   - **Key/Value/Extra (KVE) search:** These search options is specifically designed for querying using multiple metadata entries simultaneously. You can access these search options via the {{:rdms:webapp:wiki_search_avu_1.png?nolink&20|}} button at the top of the search interface.
  
-You can change between these searche options by selecting the respective tab from the top of the search interface. Below you will find more examples demonstrating how to utilize these searches for both simple and more complex quearies in the RDMS.+You can switch between these search options by selecting the respective tab from the top of the search interface. Below you will find more examples demonstrating how to utilize these searches for both simple and more complex queries in the RDMS.
  
 ==== Tabbed Searches ==== ==== Tabbed Searches ====
Line 29: Line 29:
  
 ==== Saved Searches ==== ==== Saved Searches ====
-Additionally, it is possible to save searches for future use. To save a search, click the {{:rdms:webapp:rdms_save_search.png?nolink&20|}} button in the search view after you specified the desired search terms (see below for examples on how to use the search). +Additionally, it is possible to save searches for future use. To save a search, click the {{:rdms:webapp:rdms_save_search.png?nolink&20|}} button in the search view after you specified your desired search terms (see below for examples of how to use the search). 
  
 {{ :rdms:webapp:rdms_search_save.png?direct&800 |}} {{ :rdms:webapp:rdms_search_save.png?direct&800 |}}
Line 42: Line 42:
  
 ==== Sub-Searches ==== ==== Sub-Searches ====
-Whenever a search result is found, the RDMS search allows to execute subsequent searches inside the desired location. This feature is especially helpful when the first search finds a respective folder and a consecutive search inside this folder needs to be executed. Here's how to utilize sub-searches:+Whenever a search result is found, the RDMS search allows you to execute subsequent searches inside the desired location. This feature is especially helpful when the first search finds a  folder and a consecutive search inside this folder needs to be executed. Here's how to utilize sub-searches:
  
 First find the folder in which you want to search via the initial search, then click the magnifying glass button next to the folder name. First find the folder in which you want to search via the initial search, then click the magnifying glass button next to the folder name.
Line 62: Line 62:
 ===== Using RDMS Search ===== ===== Using RDMS Search =====
 ==== General Search ==== ==== General Search ====
-To execute a basic search for data, you can specify the search term in the top-right box in the search engine and then execute the search by hitting the magnifying glass symbol:+To perform a basic search for data, you can specify the search term in the top-right box in the search engine and then execute the search by clicking on the magnifying glass symbol:
  
 {{ :rdms:webapp:rdms_search_simple_1.jpg?direct&900 |}} {{ :rdms:webapp:rdms_search_simple_1.jpg?direct&900 |}}
Line 68: Line 68:
 This will search for the specified search term using the standard search parameters.  This will search for the specified search term using the standard search parameters. 
  
-To make the search results more precise, you can adjust the search criteria. For example, you can specify that just data created in a certain time frame should be shown, that you want to see files only, that just a certain RDMS path should be search, and much more.+To make the search results more precise, you can adjust the search criteria. For example, you can specify if you want only data created within a certain time frame to be shown, to see files only, to search only certain RDMS paths, and much more.
  
 {{ :rdms:webapp:rdms_search_simple_2.png?direct&900 |}} {{ :rdms:webapp:rdms_search_simple_2.png?direct&900 |}}
Line 79: Line 79:
  
 **Note:** **Note:**
-  * This simple search will display results where the search term is found at various places. For example, it will find results where the search term is in the file name, but also where it is found in the metadata. +  * This simple search will display results where the search term is found at various locations. For example, it will find results where the search term appears in the file name, as well as where it is found in the metadata. 
  
-==== Search using Metadata ====+==== Search Using Metadata ====
  
 While the general search view can be also used to search based on metadata, the separate key/value/extra search was specifically designed to facilitate searches/queries that use multiple, combined metadata values as the search criteria.  While the general search view can be also used to search based on metadata, the separate key/value/extra search was specifically designed to facilitate searches/queries that use multiple, combined metadata values as the search criteria. 
Line 100: Line 100:
 In this example, we used a single KVE as search criteria (Key: "Author", Value: "John Doe") which resulted in four files being found by the search.  In this example, we used a single KVE as search criteria (Key: "Author", Value: "John Doe") which resulted in four files being found by the search. 
  
-If we now introduce another KVE (Key: "Date", Value: "today") and repeat the search we get:+If we now introduced another KVE (Key: "Date", Value: "today") and repeat the search we get:
  
 {{ :rdms:webapp:rdms_search_kve_3.png?direct&800 |}} {{ :rdms:webapp:rdms_search_kve_3.png?direct&800 |}}
  
-So instead of seeing the four results as before, we now just get two of the previous results shown. This results from the fact that the remaining two entries had an ''Author: John Doe'' metadata entry, but not a ''Date: today'' entry. Accordingly, they are removed from the second search. +Instead of seeing the four results as before, we now just get two of the previous results shown. This results from the fact that the remaining two entries had an ''Author: John Doe'' metadata entry, but not a ''Date: today'' entry. As a result, they are removed from the second search. 
  
-If a third criteria is introduce even less results are shown:+If a third criterion is introduced even less results are shown:
  
 {{ :rdms:webapp:rdms_search_kve_4.png?direct&800 |}} {{ :rdms:webapp:rdms_search_kve_4.png?direct&800 |}}
Line 115: Line 115:
 ==== Using Fast Search ==== ==== Using Fast Search ====
  
-Next to the search function explained so far, the RDMS also has a fast search that can be used to make simple searches from any location in the RDMS. +In addition to the search function explained so far, the RDMS also has a fast search that can be used to make simple searches from any location in the RDMS. 
  
 To access the fast search, simply click on the UG logo in the top-left corner of the web interface: To access the fast search, simply click on the UG logo in the top-left corner of the web interface:
Line 126: Line 126:
 ==== More Advanced Search Syntax ==== ==== More Advanced Search Syntax ====
  
-While it was shown above how files and folders can be found based on different parameters (name, metadata, etc.), and it was also shown that search criteria can be adjusted via the available menu to fine-tune search results, we will elaborate here which additional tricks you can use in the search syntax. +The examples above showed how to find files and folders based on different parameters (name, metadata, etc.). It was also shown that search criteria can be adjusted via the available menu to fine-tune search results, we will elaborate here advanced search syntax for further customization.
  
-When you specify a search term in the field, it is important to know that you can also used additional wildcard cards in your query. These wildcard characters are+When you specify a search term in the field, it is important to know that you can also use additional wildcard characters in your query. These wildcard characters are:
  
-  * ''_'': The underscore is the wildcard for **one sign**. So searching for "Device_-Data" will find results for "DeviceA-Data", "DeviceB-Data", and so on, but no result for "DeviceXXX-Data"+  * ''_'': The underscore is the wildcard for **one character**. So searching for "Device_-Data" will find results for "DeviceA-Data", "DeviceB-Data", and so on, but not for "DeviceXXX-Data"
-  * ''%'': The percentage sign is the wildcard for an **undefined** amount of signs. So searching for "Device%-Data" will find the same results as described for "Device_-Data", but additionally also "DeviceXXX-Data".+  * ''%'': The percentage sign is the wildcard for an **undefined** number of characters. So searching for "Device%-Data" will find the same results as described for "Device_-Data", but additionally also "DeviceXXX-Data".
  
-It is important to note that this wildcard characters can be of course also used for the search based on metadata. +Note that these wildcard characters can also be used for the search based on metadata. 
  
-To see how we can use this wildcards for further filtering of our query, take a look at this example where we first searched for "Device" which gave us in this case six similarly, but still slightly differently named folders:+To see how you can use these wildcards for further filtering of your query, take a look at this example where we first searched for "Device". This search yielded six similarly named folders, each with slight variations:
  
 {{ :rdms:webapp:rdms_search_syntax_1.jpg?direct&900 |}} {{ :rdms:webapp:rdms_search_syntax_1.jpg?direct&900 |}}
  
-Assuming that we now just want to find the configuration data for all the devices 1-3, we could use a search term like "Device%_Configuration_Data" which will yield:+Assuming that we now just want to find the configuration data for all devices 1-3, we could use a search term like "Device%_Configuration_Data" which will yield:
  
 {{ :rdms:webapp:rdms_search_syntax_2.png?direct&900 |}} {{ :rdms:webapp:rdms_search_syntax_2.png?direct&900 |}}
Line 145: Line 145:
 As we used the ''%'' wildcard character in this example, this would also find results like "Device312_Configuration_Data" or "DeviceBuilding4_Configuration_Data". As we used the ''%'' wildcard character in this example, this would also find results like "Device312_Configuration_Data" or "DeviceBuilding4_Configuration_Data".
  
-In another example, we want to find boththe analytical and configuration data folder for our "Device1", but we do not want to get the results for the other devices. This could be achieved with a query of the form "Device1_%_Data" as seen here:+In another example, suppose we want to find both the analytical and configuration data folders for "Device1", but we do not want results for the other devices. This could be achieved with a query of the form "Device1_%_Data" as seen here:
  
 {{ :rdms:webapp:rdms_search_syntax_3.png?direct&900 |}} {{ :rdms:webapp:rdms_search_syntax_3.png?direct&900 |}}
  
-Next to the wildcard characters, it is also possible to use ''&'' and ''|'' in the search will will act as "and" and "or" operators, respectively.  +In addition to the wildcard characters, it is also possible to use ''&'' and ''|'' in the search, which act as "and" and "or" operators, respectively.  
-Let's assume that you want to find the results that contain either "Device1" or "Device2" in their name. A search query for this could use the "or" statement to connect both terms like "Device1 | Device2" which will yield:+Let's assume that you want to find results that contain either "Device1" or "Device2" in their name. A search query for this can use the "or" statement to connect both terms like "Device1 | Device2" which will yield:
  
 {{ :rdms:webapp:rdms_search_syntax_4.png?direct&900 |}} {{ :rdms:webapp:rdms_search_syntax_4.png?direct&900 |}}
  
 **Note:**\\ **Note:**\\
-As seen above, we can use wildcards to further fine-tune our search results. With the other settings that are available in the search interface as well as with the possibility to use metadata, this offers a lot of capability to modify searches to yield our desired results. +As seen above, you can use wildcards to further fine-tune your search results. With other settings that are available in the search interface together with the possibility to use metadata, these advanced search techniques offer a lot of capability to obtain precise results tailored to your specific requirements. 
 + 
 +==== Filtering Your Search Results ==== 
 + 
 +In some cases, your search parameters might return a pool of results that is too large to find what you are looking for. As shown in the figure below, the RDMS Web interface also warns you when the number of results exceeds 10.000: 
 + 
 +{{ :rdms:search:sfilter_1.png?direct&800 |}} 
 + 
 +To fix this, you might want to redefine your search parameters, as the Web interface suggests. However, this can sometimes result in having to wait a long time for a search to run again. Alternatively, you might want to filter the results you have already obtained. To do so, type an additional keyword in the bar highlighted by the red box in the figure below. 
 + 
 +{{ :rdms:search:sfilter_2.png?direct&800 |}} 
 + 
 +With the new keyword added, the RDMS Web interface will show you only the results which contain the keyword. Thus you can reduce the amount of displayed results without running a new search. The figure below shows what the filtering looks like. 
 + 
 +{{ :rdms:search:sfilter_3.png?direct&800 |}} 
 + 
 +**Note:** You are filtering the results rather than redefining the search parameters. This means that should you save the search and run it again, you might encounter too many results again. Each time you run that specific search, you will need to manually filter the results. We recommend that for frequently performed searches, you redefine the parameters once you have identified the optimal keyword combination through filtering. 
 + 
 + 
 +==== Exporting Search Results (Report Generation) ==== 
 + 
 +The RDMS also comes with in-built functionality to export search results in formats suitable for report. 
 + 
 +To do so, first execute your search. Once the search results are displayed, you can select one of the available report functions: 
 + 
 +  * **Create report**: This function creates a set of JSON files containing the search results. From these files, you can then create the final report.  
 +  * **Export search results as...**: This function directly exports the search results in a presentable format that you can choose (currently *.csv)  
 + 
 +{{ :rdms:webapp:rdms_report_1.png?direct&800 |}} 
 + 
 +=== Using the Create Report Function === 
 + 
 +If you select "Create report", a dialog window will open. Here you can specify a name for your report and select if the intermediately created JSON files should be encrypted. Since the JSON file also contains the metadata of the search results, along with your search parameters, you might want to choose to encrypt it to protect potentially sensitive information.   
 + 
 +{{ :rdms:webapp:rdms_report_new_1.png?direct&400 |}} 
 + 
 +After confirmation, you can find your report by selecting the report button ({{:rdms:webapp:rdms_report_button.png?direct&25|}}) from the left navigation bar. From within the report view, you can now select to either create your final report in CSV format, view/download the JSON files containing your search results/parameters, or to delete one or more reports. 
 + 
 +{{ :rdms:webapp:rdms_report_new_2.png?direct&800 |}} 
 + 
 +If you select the option to create your final report for export, a new window will open. The first option in that window will ask you for the place to save your report file. That location is where you will be able to retrieve the final report from. The rest of the window asks you to select which part of the results you want to be included in the report (data, metadata, and/or permissions).  
 + 
 +{{ :rdms:webapp:rdms_report_new_3.png?direct&600 |}} 
 + 
 +After confirmation, you can download your report in CSV format from the specified RDMS location. 
 + 
 +{{ :rdms:webapp:rdms_report_6.png?direct&800 |}} 
 + 
 +**Notes**: 
 +  * While it is not mandatory to select encryption for the created JSON files with your search results/parameters, it should be done if exported results contain sensitive data. Also in other cases, having encryption selected is recommended as good practice. 
 +  * If you did not have any user-defined metadata, there will still be a set of system defined metadata which are added by default ("Data type", "Size", "Creator", "Modified"). 
 + 
 +=== Using Export Search Results As Function === 
 + 
 +The "Export search results as..." function allows you to directly create an exportable CSV from your search results. If you select this option from the search view, you will be asked where your exportable CSV should be saved. Afterwards, you are asked to select the export type (data, metadata and/or permissions), and to specify a name for your export (a default value is automatically filled in).  
 + 
 +{{ :rdms:webapp:rdms_report_new_4.png?direct&500 |}} 
 + 
 +Afterwards, you can download your report in CSV format from the selected RDMS location. 
 + 
 +**Notes**: 
 +  * The "Export search results as..." function is **a shortcut way** to generate an exportable CSV of your search results, and as such, **it does not allow you to directly select which metadata to extract** if metadata is included in your report. Instead all metadata is exported by default. 
 +  * Choosing the "Export search results as..." function will also create JSON files in the background. These files contain your search results and parameters. You can access them via the report button ({{:rdms:webapp:rdms_report_button.png?direct&25|}}) in the left navigation bar. By doing so, you can create further reports from these intermediate JSON files, and you can fine-tune the export parameters (e.g. which metadata to include in the CSV).  
 +  * The "Export search results as..." will automatically use encryption for the intermediate JSON files that are created.   
 + 
 +=== Importing Report Files in Spreadsheet Tools === 
 + 
 +The exported CSV reports can be important in spreadsheet tools (e.g. MS Excel, LibreOffice) for visualization and further processing. 
 +  
 +It is important to note that the CSV reports in the RDMS use **tab as field delimiter**!  
 + 
 +Please be sure that you select this during import in your spreadsheet tool as it will otherwise break the formatting of your import.  
 + 
 +Also note that RDMS metadata fields can be multi-line fields. In cases where this causes problem when exporting the CSV to another tool and where it is desired for other reasons, it is possible to automatically replace newlines in the exported CSV. To do so, simply select a replace character during report creation  
 +