Author: Andy Tauro, Performance Architects
Recently somebody asked me, “Is there a way to limit data loads to Oracle Essbase databases to ensure that only Level 0 or leaf members are loaded?” Essbase itself does not have a constraint that will disallow a user from loading data to a non-leaf member, assuming that the said user has the necessary privileges to load data to the database. In the case of the Essbase BSO (Block Storage Option), the data gets loaded just fine, and then will get replaced when the database is aggregated. In the case of the Essbase ASO (Aggregate Storage Option), one gets presented with a (very) polite warning saying that data loads to “summary” level members will be ignored.
So then why the need to limit this? The simplest reason may be to reduce the amount of data records being sent to Essbase; in other words, reduce waste to increase efficiency. Or, the less records loaded, the faster the data load. There also may be a bit more concerning reason: the data generation process is not creating records at the right level of detail. In this case, if the data is loaded to a non-Level 0 intersection, and no data is loaded to the Level 0 member, the data in the parent intersection can get wiped out when the child is aggregated up, resulting in loss of that data point.
Usually, when we implement an Essbase application, it is more than a simple database. There is a process built around Essbase to gather, process and load the data, and, of course, to ensure that the right level of security is assigned to the data. In other words, what we call an ETL (Extract, Transform, Load) process (or ELT, depending on who you’re talking with in the data integration arena!). Typically, this kind of process is built around using a user account that has enough privileges to perform both the data load as well as the application of the correct security filters. Such a user account would require at least “Application Manager” (although more often “Administrator”) status and would also have access to write data to any part of the database (leaf as well as non-leaf). However, when such processes are built, there is usually an appropriate mechanism put in to perform validations on the data, including to ensure that only leaf-level records are being generated.
When the ETL process is not that elaborate (or even when it is elaborate and there still is a need to avoid such erroneous data loads), there is a simpler way to trap such “bad” records: create a user account used only for data loads and give this account “write” privileges to just leaf members, not anything higher than that. This way, if the data load attempts to load a “bad” record, Essbase will throw a security violation error for the record, which can be caught by the ETL process to trigger appropriate action.
Such a filter only needs to restrict one dimension, so can be as simple as a ‘Write’ on ‘@LEVMBRS (Product,0)’ and ‘Read’ on ‘@IDESCENDANTS (Product)’. The error messages will be trapped in the error file, something to the effect of:
\\ Incorrect Access To Store Record
“100” “New York” “Jan” “Actual” 678 271 94 51 0 2101 644 2067
The drawback of this approach is that it does not indicate the exact dimension/member that is the cause of the failure, but at least indicates the record that is bad. If the filter is simple enough to lock down only one dimension, then we just need to look at the member for that dimension in the “bad” record (in the above case “100”). The other drawback is the need for a dedicated user account for data loads, in addition to the account created for the rest of the ETL process like dimension and security refresh.
All said and done, there may be many creative ways to get to the same goal, making life better with Essbase. Which one you pick depends on what works for your implementation. Having a knowledgeable implementation partner helps. Need some help with your implementation or with coming up with a solution, or do you just want to comment on this idea? Send us a note at communications@performancearchitects.com. We would love to hear from you.
Cheers!
