# The Insight Engine **URL:** https://forum.squirro.com/c/the-insight-engine/7.md [Latest](https://forum.squirro.com/latest.md) · [Categories](https://forum.squirro.com/categories.md) · [Tags](https://forum.squirro.com/tags.md) --- ## [The Insight Engine](https://forum.squirro.com/t/the-insight-engine/25)
**Author:** [@Lauren](https://forum.squirro.com/u/Lauren)\ **Replies:** 0
In using or simply discovering our software, you will most likely have questions and some feedback. Maybe you have a specific technical query or maybe you have tested a feature and want to understand it more, exchange t… --- ## [Is there a way to override the 'upload date' of a document?](https://forum.squirro.com/t/is-there-a-way-to-override-the-upload-date-of-a-document/302)
**Author:** [@TalitaAnthonio](https://forum.squirro.com/u/TalitaAnthonio)\ **Replies:** 3\ **Last updated:** [July 29, 2024, 5:41am UTC](https://forum.squirro.com/t/is-there-a-way-to-override-the-upload-date-of-a-document/302 "2024-07-29T05:41:43Z")
Hi Squirro Team, I am currently working on a project where I am building a squirro dashboard for a custumer. The custumer asked me if it would be possible to use the origin date as value for ‘sort by date’, instead of … --- ## [Format for uploading a workflow with custom pipelet step](https://forum.squirro.com/t/format-for-uploading-a-workflow-with-custom-pipelet-step/290)
**Author:** [@nquinn](https://forum.squirro.com/u/nquinn)\ **Replies:** 6\ **Last updated:** [October 10, 2023, 8:51pm UTC](https://forum.squirro.com/t/format-for-uploading-a-workflow-with-custom-pipelet-step/290 "2023-10-10T20:51:52Z")
Hi, I’m trying to use the new pipeline workflow API to upload a new pipeline workflow that includes a custom pipelet I built. I can’t find any example formats in the docs that describe what the value of type should be w… --- ## [Uploading data with DocumentUploader() creates a new pipeline](https://forum.squirro.com/t/uploading-data-with-documentuploader-creates-a-new-pipeline/294)
**Author:** [@TalitaAnthonio](https://forum.squirro.com/u/TalitaAnthonio)\ **Replies:** 4\ **Last updated:** [September 7, 2023, 2:52pm UTC](https://forum.squirro.com/t/uploading-data-with-documentuploader-creates-a-new-pipeline/294 "2023-09-07T14:52:25Z")
Hi, I am trying to upload a collection of documents using squirro\_client. Whenever I do this, I am unable to pick a pipeline using ‘pipeline\_workflow\_id’. Instead, I see a new pipeline in the user-interface called ‘Ite… --- ## [Changing project picture results in project error](https://forum.squirro.com/t/changing-project-picture-results-in-project-error/273)
**Author:** [@daren.sin](https://forum.squirro.com/u/daren.sin)\ **Replies:** 1\ **Last updated:** [May 10, 2023, 1:28pm UTC](https://forum.squirro.com/t/changing-project-picture-results-in-project-error/273 "2023-05-10T13:28:36Z")
On the Squirro UI, I changed my project picture into the following: https://lh3.googleusercontent.com/S018GEmYCdDQgxSp-\_rb2Ic7L-YdywTq8OJIW8kYuKi7oOW-J1g\_eyDq4Hbw2uMf8xSwa0-ztT8DSttMMcTxfyT3C0wMe3aCYc4g6n9zn7WsMk1hXGb8jq… --- ## [Limit length of abstract in search results using query endpoint with hit highlighting](https://forum.squirro.com/t/limit-length-of-abstract-in-search-results-using-query-endpoint-with-hit-highlighting/256)
**Author:** [@kerfle](https://forum.squirro.com/u/kerfle)\ **Replies:** 4\ **Last updated:** [January 31, 2023, 2:11pm UTC](https://forum.squirro.com/t/limit-length-of-abstract-in-search-results-using-query-endpoint-with-hit-highlighting/256 "2023-01-31T14:11:54Z")
Hi everyone, I’m currently using the query endpoint in our python code to deliver search results with hit highlighting to our React frontend. We want to display example text with the hit highlights in our search results… --- ## [Ways to define images when creating communities](https://forum.squirro.com/t/ways-to-define-images-when-creating-communities/246)
**Author:** [@Nathan](https://forum.squirro.com/u/Nathan)\ **Replies:** 2\ **Last updated:** [December 19, 2022, 9:39am UTC](https://forum.squirro.com/t/ways-to-define-images-when-creating-communities/246 "2022-12-19T09:39:23Z")
Hello all, I have a question regarding image-sourcing options when it comes to creating communities. What is the preferred way to present image information for communities within a CSV or XLSX file, and are there multi… --- ## [Squirro Docs Redesign, Content Migration from the Wiki, and Recently Added Content](https://forum.squirro.com/t/squirro-docs-redesign-content-migration-from-the-wiki-and-recently-added-content/245)
**Author:** [@Nathan](https://forum.squirro.com/u/Nathan)\ **Replies:** 0\ **Last updated:** [December 7, 2022, 4:22pm UTC](https://forum.squirro.com/t/squirro-docs-redesign-content-migration-from-the-wiki-and-recently-added-content/245 "2022-12-07T16:22:22Z")
Hello all, The Squirro Docs site has undergone some changes lately, and we wanted to share a few updates: :squirro: The Squirro Docs redesign is now live, and over time we will be updating all existing content to ali… --- ## [Extending Items Table (Facets Table) with an action button](https://forum.squirro.com/t/extending-items-table-facets-table-with-an-action-button/232)
**Author:** [@aunaz](https://forum.squirro.com/u/aunaz)\ **Replies:** 7\ **Last updated:** [November 7, 2022, 1:08pm UTC](https://forum.squirro.com/t/extending-items-table-facets-table-with-an-action-button/232 "2022-11-07T13:08:51Z")
I try to extend the items table with a custom action button that opens a modal with content. Also, I want to open the item details modal on row click. This is the default behavior of the items table. The problem is my … --- ## [Structured vs. Unstructured Data](https://forum.squirro.com/t/structured-vs-unstructured-data/223)
**Author:** [@Nathan](https://forum.squirro.com/u/Nathan)\ **Replies:** 2\ **Last updated:** [October 4, 2022, 11:52pm UTC](https://forum.squirro.com/t/structured-vs-unstructured-data/223 "2022-10-04T23:52:56Z")
Hi all, Would anyone care to share a definition of structured versus unstructured data in the Squirro context, specifically mentioning how it relates to the data loading and data processing stages of a project? --- ## [Data loader modularization](https://forum.squirro.com/t/data-loader-modularization/214)
**Author:** [@nquinn](https://forum.squirro.com/u/nquinn)\ **Replies:** 2\ **Last updated:** [September 15, 2022, 2:26pm UTC](https://forum.squirro.com/t/data-loader-modularization/214 "2022-09-15T14:26:08Z")
Hi, I have been using an adapted version of the SharePoint data loader to extract files from SharePoint sites, but now I need to add the capability to extract SharePoint list items instead of files in drives. The logic … --- ## [Partner Question: What is a Catalyst?](https://forum.squirro.com/t/partner-question-what-is-a-catalyst/208)
**Author:** [@Lauren](https://forum.squirro.com/u/Lauren)\ **Replies:** 1\ **Last updated:** [August 31, 2022, 9:45am UTC](https://forum.squirro.com/t/partner-question-what-is-a-catalyst/208 "2022-08-31T09:45:18Z")
In our recent technical office hour a partner asked - What is a Catalyst? --- ## [Date error when importing JSON file](https://forum.squirro.com/t/date-error-when-importing-json-file/205)
**Author:** [@bdx](https://forum.squirro.com/u/bdx)\ **Replies:** 6\ **Last updated:** [August 24, 2022, 7:16am UTC](https://forum.squirro.com/t/date-error-when-importing-json-file/205 "2022-08-24T07:16:00Z")
Hi there, I am having an issue with the importer. I am loading a file manually (a JSON). The date labels work correctly for other files. This particular file however does not index any of the items and in the logs, prod… --- ## [Completing large data loads](https://forum.squirro.com/t/completing-large-data-loads/201)
**Author:** [@nquinn](https://forum.squirro.com/u/nquinn)\ **Replies:** 1\ **Last updated:** [August 19, 2022, 8:05pm UTC](https://forum.squirro.com/t/completing-large-data-loads/201 "2022-08-19T20:05:08Z")
Hi, I have a custom CLI data loader for SharePoint that has to load ~45gb across ~6k files, and the squirro\_data\_load script ran for about 80 minutes, uploading ~3.7k of those files, then hit some kind of server-side ti… --- ## [Question from Trainee - Gmail Account - Data Source - Incorrect Connection](https://forum.squirro.com/t/question-from-trainee-gmail-account-data-source-incorrect-connection/199)
**Author:** [@Lauren](https://forum.squirro.com/u/Lauren)\ **Replies:** 1\ **Last updated:** [August 18, 2022, 2:25pm UTC](https://forum.squirro.com/t/question-from-trainee-gmail-account-data-source-incorrect-connection/199 "2022-08-18T14:25:33Z")
I just tried to connect to my gmail account as a data source but I got an error. I followed unsafe way under the advanced menu to connect. Can you please advise me on this connection and using it as a data source. Thanks … --- ## [Running Squirro in Azure](https://forum.squirro.com/t/running-squirro-in-azure/180)
**Author:** [@cmarino](https://forum.squirro.com/u/cmarino)\ **Replies:** 4\ **Last updated:** [July 11, 2022, 2:36pm UTC](https://forum.squirro.com/t/running-squirro-in-azure/180 "2022-07-11T14:36:44Z")
Does Squirro have a recommendation/best practice for running in Azure? Is it best to set up a VM and install on it? Thanks, Chris --- ## [Batch-size vs. source-batch-size arguments](https://forum.squirro.com/t/batch-size-vs-source-batch-size-arguments/187)
**Author:** [@maximilian.hoeschler](https://forum.squirro.com/u/maximilian.hoeschler)\ **Replies:** 4\ **Last updated:** [July 7, 2022, 12:57pm UTC](https://forum.squirro.com/t/batch-size-vs-source-batch-size-arguments/187 "2022-07-07T12:57:23Z")
Hello dear squirro community, during my work on a custom dataloader, I have observed an unexpected behaviour concerning the batch-size and source-batch-size arguments when using a load.sh script to manually load extract… --- ## [How to map 'created at' field with a custom date format](https://forum.squirro.com/t/how-to-map-created-at-field-with-a-custom-date-format/182)
**Author:** [@nquinn](https://forum.squirro.com/u/nquinn)\ **Replies:** 6\ **Last updated:** [June 29, 2022, 3:55pm UTC](https://forum.squirro.com/t/how-to-map-created-at-field-with-a-custom-date-format/182 "2022-06-29T15:55:05Z")
I am trying to load a CSV file using the built-in CSV data loader, and want to map a particular column to the created\_at field. This column is formatted like so: ‘2/11/2022 12:00:00 AM’. When I run the data loader with … --- ## [Removing Individual Countries from Communities](https://forum.squirro.com/t/removing-individual-countries-from-communities/150)
**Author:** [@manutej](https://forum.squirro.com/u/manutej)\ **Replies:** 5\ **Last updated:** [June 27, 2022, 1:55pm UTC](https://forum.squirro.com/t/removing-individual-countries-from-communities/150 "2022-06-27T13:55:00Z")
Hello, I have a newsreader that uses countries tagged in the document for a Squirro News Reader. For my use case, I only want to focus on specific countries and therefore want to remove the other countries. I keep runni… --- ## [Partner Question - When writing a data loader plugin, what do you do with large binary data?](https://forum.squirro.com/t/partner-question-when-writing-a-data-loader-plugin-what-do-you-do-with-large-binary-data/169)
**Author:** [@conor.leddy](https://forum.squirro.com/u/conor.leddy)\ **Replies:** 4\ **Last updated:** [June 21, 2022, 6:11am UTC](https://forum.squirro.com/t/partner-question-when-writing-a-data-loader-plugin-what-do-you-do-with-large-binary-data/169 "2022-06-21T06:11:11Z")
Question from Partner: When writing a data loader plugin, what do you do with large binary data? Is there a way to hand off a file pointer, or do I just read in 10MB of PDF and stuff it in a JSON element, or what? Joh… --- ## [Are there some resources I can use to resolve/delete things creating high server load?](https://forum.squirro.com/t/are-there-some-resources-i-can-use-to-resolve-delete-things-creating-high-server-load/171)
**Author:** [@manutej](https://forum.squirro.com/u/manutej)\ **Replies:** 2\ **Last updated:** [June 21, 2022, 5:58am UTC](https://forum.squirro.com/t/are-there-some-resources-i-can-use-to-resolve-delete-things-creating-high-server-load/171 "2022-06-21T05:58:43Z")
Hello, I have recently overloaded one of my Squirro instances with too much articles and machine learning jobs. This means that I am reaching the memory limit of my instance with the level of resources I have on the ser… --- ## [Customer Inquiry - Cognitive Search Offering - Squirro](https://forum.squirro.com/t/customer-inquiry-cognitive-search-offering-squirro/173)
**Author:** [@Lauren](https://forum.squirro.com/u/Lauren)\ **Replies:** 1\ **Last updated:** [June 9, 2022, 10:13am UTC](https://forum.squirro.com/t/customer-inquiry-cognitive-search-offering-squirro/173 "2022-06-09T10:13:45Z")
:question: Does Squirro Cognitive Search provide a single search of all content across time, space, and language? --- ## [Delete item in SharePoint dataloader](https://forum.squirro.com/t/delete-item-in-sharepoint-dataloader/167)
**Author:** [@kerfle](https://forum.squirro.com/u/kerfle)\ **Replies:** 2\ **Last updated:** [May 31, 2022, 11:08pm UTC](https://forum.squirro.com/t/delete-item-in-sharepoint-dataloader/167 "2022-05-31T23:08:04Z")
Hi all, We are adapting the SharePoint dataloader for a client and they want any items deleted in SharePoint to be correspondingly deleted in Squirro. The dataloader uses a delta API to retrieve new or edited items, inc… --- ## [Field Level Security in Squirro](https://forum.squirro.com/t/field-level-security-in-squirro/163)
**Author:** [@cmarino](https://forum.squirro.com/u/cmarino)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 1:45pm UTC](https://forum.squirro.com/t/field-level-security-in-squirro/163 "2022-05-16T13:45:04Z")
Hi all - We have this requirement from a proposed client. Does Squirrro handle field-level security? “Proposed solution must be able to secure data on the level of the field, so one employee may see a field in a documen… --- ## [AI Studio - How to delete published models?](https://forum.squirro.com/t/ai-studio-how-to-delete-published-models/151)
**Author:** [@Moritz](https://forum.squirro.com/u/Moritz)\ **Replies:** 4\ **Last updated:** [April 27, 2022, 10:23am UTC](https://forum.squirro.com/t/ai-studio-how-to-delete-published-models/151 "2022-04-27T10:23:39Z")
hey, I have published a number of models which I am using the Squirro pipeline. Now I have noticed that there are some models with similar names and I would like to delete some models which I do not need anymore. How d… --- ## [Exception in RSS feed dataloader plugin](https://forum.squirro.com/t/exception-in-rss-feed-dataloader-plugin/143)
**Author:** [@daren.sin](https://forum.squirro.com/u/daren.sin)\ **Replies:** 2\ **Last updated:** [April 26, 2022, 6:44am UTC](https://forum.squirro.com/t/exception-in-rss-feed-dataloader-plugin/143 "2022-04-26T06:44:57Z")
Hi, I encountered this exception with my RSS feed dataloader plugin. I supposed feed\_plugin should be in the \_global folder, but it is going to squirro-0 instead. I created this data source using the RSS feed dataloader … --- ## [Cannot see output in a lib python file](https://forum.squirro.com/t/cannot-see-output-in-a-lib-python-file/147)
**Author:** [@haarisjalal](https://forum.squirro.com/u/haarisjalal)\ **Replies:** 6\ **Last updated:** [April 19, 2022, 1:53pm UTC](https://forum.squirro.com/t/cannot-see-output-in-a-lib-python-file/147 "2022-04-19T13:53:14Z")
I have created a python file inside the lib folder. However, I cannot see the output of the log file. Are there any additional steps that I need to take to see the output of the log file. restarting the frontend is not h… --- ## [Populating Synonym List Programmatically](https://forum.squirro.com/t/populating-synonym-list-programmatically/145)
**Author:** [@cmarino](https://forum.squirro.com/u/cmarino)\ **Replies:** 2\ **Last updated:** [April 19, 2022, 1:21pm UTC](https://forum.squirro.com/t/populating-synonym-list-programmatically/145 "2022-04-19T13:21:10Z")
I found this page on implementing Synonyms in Squirro very helpful. https://squirro.atlassian.net/wiki/spaces/DOC/pages/2015199283/Using+Synonyms Is there a programmatic way to populate the terms in the synonym list (v… --- ## [High Charts Filter Clearing Issue](https://forum.squirro.com/t/high-charts-filter-clearing-issue/142)
**Author:** [@haarisjalal](https://forum.squirro.com/u/haarisjalal)\ **Replies:** 1\ **Last updated:** [April 11, 2022, 7:24am UTC](https://forum.squirro.com/t/high-charts-filter-clearing-issue/142 "2022-04-11T07:24:46Z")
Hi there, We are using high charts in two of our widgets. Once, we select something there, filters are applied. If we select something again, an additional filter is applied as well. However, when we remove one of the … --- ## [Change max\_connections value for AWS RDS managed service for mariadb](https://forum.squirro.com/t/change-max-connections-value-for-aws-rds-managed-service-for-mariadb/141)
**Author:** [@chiara](https://forum.squirro.com/u/chiara)\ **Replies:** 3\ **Last updated:** [April 8, 2022, 12:23pm UTC](https://forum.squirro.com/t/change-max-connections-value-for-aws-rds-managed-service-for-mariadb/141 "2022-04-08T12:23:08Z")
Hi, we have a Squirro installation on AWS which is setup to use the MariaDB managed service from RDS. The RDS instance was recently down-scaled to db.t3.small with 2GB RAM. After that change, we are sometimes seeing e… [Next page](https://forum.squirro.com/c/the-insight-engine/7.md?page=1)