The latest Squirro Release 3.7.0 is now generally available on our mirror. In order to upgrade your existing Squirro installation or install a new version of Squirro, please head over to the Getting Squirro page for relevant instructions.
Latest SquirroClient is also now available as usual on PyPI.
Squirro 3.7.0 was released Jan 11, 2023.
What’s New
- You can now install and run docker on production instances.
- A link to the Squirro Monitoring space was added to the Spaces Menu, giving administrative users quick access to view their project’s activity and data ingestion logs.
- The PDF-OCR step now includes optional Confidence scoring, which can be enabled at some performance cost.
- Added a Data Loader option to delete unwanted additional fields by clicking the trash icon.
- Added a Created At column to show the creation date for proximity rules within AI Studio.
- Added a new libNLP step to call spaCy running as a Squirro NLP service.
- Created a Binary Documents pipeline for new projects.
- Added a HFQuestionAnswering processor that can run Hugging Face question-answering pipelines for inference.
Improvements
- Improved the performance of PDF document ingestion for pipeline workflows which include ML models.
- Added information about sender and recipients to the attachments items in the Exchange data loader plugin.
- React widgets will now fetch data only if they are in a visible dashboard section layer.
- Moved all libNLP steps calling external services (mlflow_maas, endpoint) to the steps subfolder external. For compatibility reasons, the top-level registration was kept, allowing steps to still be addressed via their step name.
- Upgraded React to v18.
- Clicking on a community typeahead suggestion will now redirect the user to the selected community.
- Implemented an NLP step for bulk labeling.
- Added an endpoint for creating bulk labeling.
- Added handling configuration for the bulk labeling step.
- The previous “PDF Cannot Be Displayed” error message has been replaced with the more generic “This document cannot be displayed” message and now makes a better attempt to display a working file link.
- Increased the clickable area for the subscribe button in the Communities List widget.
- Squirro now uses tika-pdf-sentences by default for speedier PDF Sentences Tokenization.
- Analyzed query tokens that might contain valid sub-tokens are now additionally re-written to perform exact phrase matching during query processing. NewYork => (“NewYork”~0 OR NewYork). This enables sub-word matching on New, and York individually relying on the configured SearchAnalyzer (subword-delimiter) - but will additionally match the exact phrase NewYork as well.
- Implemented a ML endpoint for creating ml job which automatically creates labels.
Bug Fixes
- Improved explicit error handling when a new monitoring project is created from a template, but errors out because a monitoring project exists already. (Only one monitoring project per server is allowed).
- Fixed an issue with GlobalSearch disappearing after searching or a dashboard switch.
- Fixed distorted thumbnails in the Communities Item widget.
- Fixed an issue with model templates demo view not working.
- Similarly to the rerun from index, now rerun from raw data will not remove existing items from the index if a step fails during a rerun of those items. The items will be replaced in the index only when the rerun is successful, otherwise the existing ones will remain in the index.
- Fixed several issues with entities and queries loading inside of the Items widget side panel.
- Fixed an issue with QA answer highlighting in item detail overlay when there are multiple answers for the same item.
- Fixed an issues with apiOverrides not working for React custom widgets.
- Fixed an issue with the ResetFilters widget adding the same selection multiple times.
- Fixed a bug where the dashboard menu icon disappears after clicking exit while editing dashboards.
- Added validation for pipelet’s sections order in pipeline workflows.
- Fixed the wrong position of the KEE pipelet in pipeline workflows.
- Fixed a bug with the tika-server systemd service.
- Fixed a bug where project theme and guide are not fetched after importing a project.
- Now, deleting a project will check if any other existing project uses any of the indices of the to-be-deleted project, and it will refrain from deleting any shared index in order not to break the existing project.
- Fixed a rare issue where data loader plugins with the same file names got mixed up.
- Fixed a bug where autosize stops working when toggling edit mode.
- Fixed the error (psycopg2.DatabaseError) error with status PGRES_TUPLES_OK and no message from the libpq which appeared sometimes in the ingester logs when executing pipelet steps in PostgreSQL installations.
- Fixed Pie Chart not reacting to config and cutom query changes.
- Fixed a bug with the Squirro Space button not shown after new project has been created.
- Fixed an issue with Aggregated Communities missing queries.
- Fixed a bug where the concept search params would not disappear from the url.
- Fixed entities and matching queries not working inside communities headlines item detail.
- Fixed entities list not re-rendering when unselecting the chip.
- Fixed an issue with the PDF-OCR step failing on mime-type detection.
- Community List widget no longer breaks after subscribing to more than 15 communities.
- Fixed dashboard store to URL serialization issue for persistentLinks feature.
Breaking Changes
- webshot generated thumbnail file is not added to Item files list to ensure webshot removal when item gets deleted
Installation and Upgrade
You will have to resolve at least the following config files when upgrading from Squirro 3.6.8.
/etc/squirro/common.ini
For new installations, please follow the Setup on Linux instructions.
To upgrade an existing installation, please consult the Upgrades for Squirro 3.5.3 and later guide.