Navigation in the article
- About resolver
- GS1 SK GitHub
- Choosing a method of string processing
- GS1 DL string processing in GS1 SK resolver
About resolver
In order to present and educate on GS1 standards, we created our own GS1 Digital Link (GS1 DL) resolver.
We have designed Resolver to meet our requirements and needs. In terms of the GS1 Digital Link standard, the Resolver is not GS1-Conformant. This means that it does not have all the features and functions that are defined in the GS1 DL standard.
We decided for this solution because of its simplicity. The GS1-Conformant resolver must support a number of functions that we do not currently consider necessary for our use case. For example, communicating via API with other resolvers, decompressing strings from RFID tags, or providing all available links via the Linkset standard. We do not need the above, and other, features; we have not implemented them for this reason, which has simplified and speeded up the process of creating the resolver and also deploying it for use. Other reasons were the practicality and realism of the solution. Decompression, CORS queries and link sharing by Linkset standard may be useful features for some manufacturers, but from discussions with our members, we found that they were more interested in the main feature (product link) than the other features, such as providing a link from their local resolver to the global decentralized GS1-Conformant resolver.
In addition, we have implemented features in the resolver that are not part of the GS1-Conformant resolver, but we find them useful. For example, if the resolver does not find a record for the full path of the GS1 DL string, e.g. for batch and serial number, it will move the given identifiers to a query string, which it will append to the database link for the primary identifier (e.g. HTTPS://ID.GS1SK.ORG/01/08580000000078/10/Sarza858/21/57).
For the GS1 Digital Link štandard it is mandatory to follow the GS1 DL string structure, which can be written into the QR code. Adherence to this structure is important for its functionality, e.g. to be readable at the checkout or by a handheld scanner. The processing of the GS1 DL string by the website (resolving) is covered by the GS1-Conformant Resolver Standard document as a set of recommendations, not as a mandatory processing method. The method of processing a GS1 DL string is determined by whoever owns or produces the objects marked with it.
Our resolver is hosted on the subdomain HTTPS://ID.GS1SK.ORG.
The resolver we have created serves only as an example, it does not define or limit the functionality of other resolvers.
GS1 SK GitHub
We are preparing to make our resolver available via the GitHub platform on the GS1 SK GitHub account.
Information will be updated
Choosing a method of string processing
When planning the resolver, we were deciding between two ways to search for GS1 Application Identifiers (GS1 AI) in the records:
- Getting one exact record by searching all keys.
- Retrieve all records for the primary key and then filter by key qualifiers.
Both ways are easy in case we have just 1 web page for 1 product. Then they are basically identical. The difference occurs if we have multiple web pages for 1 product, or we have a combination of 1 product : 1 web page (1:1) and 1 product : multiple web pages (1:N). For example, in addition to a product page, we have a recipe or tutorial page that we want to redirect the customer to via the 'linkType' attribute. Alternatively, if we are recalling a particular batch from sale, we can redirect the link from the product page to the 'Defective batch' page to alert the customer about the defective product.
The first method should theoretically be faster in terms of both query and processing because the query should result in a specific value according to the specified GS1 AI and the subsequent processing should be minimal. The problem arises when there is data in the barcode that is not in the database. For example, there may be a batch in the barcode, but it is not in the database because we only have one product web page independent of the batch. If we make a selection based on all attributes, the selection for the GTIN + batch combination will fail. In that case we need to repeat the selection, but only for GTIN. Alternatively, if the database supports it, make the selection with a condition that checks the existence of the batch. The complexity of this method can increase as data is added to the barcode and also as attributes are added to the 1:N relationship, where changing the value of an attribute can lead to a different page.
The second way is simpler in terms of querying the database. The database query selects all records for a given primary GTIN. Then, from the selected records, the one matching the specified primary key qualifiers is selected. If the search fails, e.g. does not find the GTIN + batch combination, it selects the default value for GTIN. As in the previous case, the complexity increases as data is added to the barcode and to the 1:N relationship.
The difference between these methods is where the selection of values occurs. In the former it is in the database query, in the latter it is in the back-end script.
Benchmark
Out of curiosity, we decided to create both and benchmarked them against 100,000 queries. Given that we are not a manufacturer and this site is educational, we found the given amount of queries to be a suitable test sample to provide us with relevant results. When deploying to a real manufacturer's site, we recommend performing measurements with a larger test sample.
We decided to perform the measurements on the 4 test strings that we expect to use most often in our examples:
- A string with a valid batch and CPV (both assigned a value in the database) and a longer starting URL path.
- A string with a batch that is not in the database and with a best before date.
- A string with the batch, CPV and serial number not in the database and with a best before date.
- A string with GTIN identifier only.
In the first three measurements, the second method, i.e., retrieving all records for the primary key and subsequent filtering, was faster by an average of 10%. In the fourth measurement, the first method, i.e., obtaining one accurate record for all keys, was faster by 0.24%. In sum, the second method is faster by 7.58% on average.
To get the most accurate results, it would be correct to take measurements for all scenarios. As mentioned before, this site is not a real manufacturer's site and therefore we decided to perform only our selected, likely, scenarios.
Selected string processing method
Based on the measurement results, we decided to use a second method of resolving GS1 strings - 'Retrieving all records for the primary key and then filtering by key qualifiers'.
Since we are not a manufacturing company and we have a well-defined, short list of products that we want to present as examples, we decided not to use a database, but a file in .json format with data in object structure. We also tried the above benchmarks on this file, which achieved 90% better time results. We attribute this large difference to the fact that we have few records in the file ( < 20), the file has few rows ( < 100 ), and the test server uses SSD storage. Using the .json format has a number of disadvantages compared to database solutions, most notably that JSON is not designed to handle large amounts of dynamic data. In our case, however, we are dealing with small amounts of static data that will only be read. There will be no writing, which avoids another disadvantage, which is the risk of 'json corruption' that can occur when data in a .json file is written and overwritten frequently. There will also be no user data input to the .json file. Relevant data will be extracted from the user data, sanitized and validated. They will then be used only as keys for comparison with the selected values. Given these facts, we concluded that the disadvantages of the .json file will not affect us much. The advantages of a static .json file are that it is easy to read and it is secure, since no writes are made to the file, the file does not contain sensitive data (all data in the file is public), and other parts of the site will not be compromised if access to the file is gained. Therefore, we opted for a JSON data structure in a .json file.
We also have a backup version of the resolver that connects to the MySQL database. In case of problems with the .json file, we can change the resolving method. We have thought of using SQLite database, but we are leaving that for future possible development of GS1 Slovakia Digital Link web site project.
GS1 DL string processing in GS1 SK resolver
The following list shows the processing sequence of the GS1 DL string on the HTTPS://ID.GS1SK.ORG website.
Identifying the requested URL on the HTTPS://ID.GS1SK.ORG website:
- The user opens the GS1 DL link: HTTPS://ID.GS1SK.ORG/01/08580000000030
- The web page router evaluates the embedded URI:
- The router recognizes GS1 DL syntax (the URL path at some position contains the primary GS1 identifier that the resolver accepts). It starts processing the GS1 DL string.
- It finds that it is a URL querying the HTTPS://ID.GS1SK.ORG website. Redirects the user to the requested page.
GS1 DL string processing in GS1 SK resolver:
- Get and sanitize the URL path.
- #1 If it exists, it separates and saves the query string from the URL path.
- Divides the URL path into fragments by slash.
- Finds out with which fragment the GS1 DL syntax starts. Looks for the primary GS1 identifier that the resolver accepts.
- Validates the value of the primary identifier:
- The value is not valid. Redirects the user to the 'Data Error' web page.
- The value is valid. Continue resolving the string.
- Attempts to retrieve the assigned values from the database. It will use the value of the primary identifier as the key.
- The value of the primary identifier does not correspond to any value in the database. Redirects the user to the 'Data not found' page.
- Finds the associated value in the database to the primary identifier. Continues resolving the string.
- #2 Determines whether the URL path contains key qualifiers at the starting index.
- It does not. Returns the default data for the given identifier. Continues in #3.
- It does. Begins processing two values according to the starting index from the URL path:
- Validates the first value (identifier key):
- The identifier is not valid (the resolver does not recognize it). It truncates the entire key qualifier URL path and puts it in the URL query string. Returns the default data for the given identifier. Continues in #3.
- The identifier is valid (the resolver accepts it). Processing continues.
- Validates the second value (the identifier value):
- The identifier value is not valid. It truncates the entire key qualifier URL path and puts it in the URL query string. Returns the default data for the given identifier. Continues in #3.
- The identifier value is valid. Processing continues.
- Attempts to retrieve the associated values from the database based on the key and value from the URL path:
- The searched values do not match any value in the database. It truncates the entire key qualifier URL path and puts it in the URL query string. Returns the default data for the given identifier. Continues in #3.
- Finds the associated values in the database to the searched values. Removes unmatched values at the same level. Moves the starting index for the URL path 2 up. Calls #2 with the new input values.
- Validates the first value (identifier key):
- #3 From the data from the previous step, get the URL to redirect according to the 'linkType' URL query attribute from the original URL.
- The original URL did not contain a 'linkType' attribute. Returns the default link. Continues in #4.
- The original URL had a 'linkType' attribute:
- The data in the resolver does not contain a value for the specified 'linkType'. Returns the default link. Continues in #4.
- The data in the resolver contains the value for the specified 'linkType'. Returns the link for the given 'linkType'. Continues in #4.
- #4 If they exist, it concatenates the URL query strings from points #1 and #2 into a single URL query string.
- #5 Concatenates the retrieved link from step #3 with the URL query string from step #4 if it exists.
- Performs a redirect to the link created in step #5.