Mixed Facet / Entities Heatmap

The Insight Engine
1

Hello,

Is there currently an easy way to set up a dashboard where one can use facets and entities in a heatmap?

For example, in the heatmap dashboard settings, you can currently only select between two facets. However, being able to see Company (facet) vs Signals (Entities) could be quite a useful way to understand which companies are strongly expressing which signal, straight from the data.

Thanks,

Manu Mulaveesala
Squirro Solutions Engineer

2 Likes
2

Hey @manutej fantastic question! Thanks for this, I’m sure this functionality to the widget could produce some meaningful insights.

Whilst there is no way to do this using the current configurations for the default Heat Map widget. There are two simple workarounds I could suggest.

You can either go ahead and extend this default widget by customising it with this functionality.

Alternatively, you can use the Squirro Client to create a new facet with your required values from the entities you wish to visualise and use this new facet to configure your heat map.

I hope that’s helpful. Don’t hesitate to come back here and share with us your solution!

2 Likes
3

I’m not convinced you can achieve this with a custom widget. Squirro does not support aggregation on entities, which would be required to achieve this.

Amin (@sciurus_vulgaris) is on the right track for the solution. Instead of some manual SquirroClient work I would however use a pipelet that does this.

1 Like
4

Ok, since @sciurus_vulgaris asked for the solution, I will share what we came up with for this situation, that can be generally applied to various other use cases as well.

The key thing to keep in mind is the following:
You have to be aware that when you are comparing an entity (sentence-based) to a facet (document based) you will have to convert (in this approach)an entity to a facet. Therefore you will lose some level of information that may have been captured by the entity. Therefore this is important to be aware of when we make facets that could be found anywhere in a document, because any entity (sentence level) will be converted to a facet at the document level. So please review the utility of this as per your needs.

That being said, here is how one can code this in a custom pipelet. In this case, we updated the enrich section of our pipelet, to be able to have this work correctly. Code was helped by @Oliver and @abdul:

    def _enrich(self, item):
        kw = item.setdefault("keywords", {})

        entities = item.get("entities", None)
        if not entities:
            return item

        signal_categories = set()
        for entity in entities:
            category = entity.get("name")
            if category:
                signal_categories.add(entity["name"])

        kw["topic"] = list(signal_categories)
        item["keywords"] = kw
        return item

Finally, you can rerun your pipelets with the following command to update all documents with some entities to have facets created for those entities:

Bash command to run this for all documents using query ‘*’

pipelet rerun --cluster CLUSTER --token TOKEN --project-id PROJECT --query '*' mypipelet.py

Please let me know if any other questions about this solution.

4 Likes