2021-06-03 | Meta | Fill In Between
These are some aspects of the current situation that cannot be changed in a way that provides enough beneft to tackle:
TepidSW is all-in on their giant, common schema. They are basing their ERP software on it, and their BI product is following along. They also appear to be getting other on-prem and cloud providers to join up. While it is true that it is quite nasty, as deep relational databases are very expensive to change, and this means, despite TepidSW's claims to the opposite, that it affects our ability to be resilient in the face of quick change and stress, a common schema across ERP and related systems has been a dream for many for fifty years. We just have to add here that extensible is not the same as resilient. Some crises require the ability to start over closer to the trunk of the tree. Regardless, TepidSW is not going away. If anything, people are accepting their marketing and embracing their products more. The virus is spreading, becoming part of the hosts it invades, and we now need to deal directly with the hybrid. Technically this means that we fill in with references to their giant common schema, create mappings (lookup tables).
Logs with key-value pairs are both commodity and proprietary. There are free and open source products as well as associated analytics. RFC 5424 provides a perfect transport via Rsyslog and others. It is a mistake to insert anything extra into these streams. This is a mistake that Triple Pub made earlier. Both Splunk and Graylog provide lookup tables. Rather than inject anything extra in the event stream itself, Triple Pub will create lookup tables.
Much of the motivation behind changes in IT will continue to be on both decomposing workstreams and automating intelligence. Few will break out of this. The holistic drum, design insight for all, will get smaller and smaller in diameter, until beating the drum has an audience of one, the drummer. We live in a DNA world.
For knowledge collaboration, HPSESS has value, but it is important to keep in mind that the outcome of the collaboration is focused on a particular model and ontology. There is no reason why the same methods couldn't be used to create different models and ontologies; however, make sure that there isn't conflation with the other aspects that are better left untouched (like putting graph information in the event streams).