Remove duplicate entities in a connector

Search
1

Hi,

I built a connector to pull in FAQs stored across a website on different pages. We have ~5 faqs at the end of each page and sometimes the same FAQ is hosted on different pages. As a result, every time I run the connector, I end up having duplicated FAQs with a different entity ID and landing page, as they are coming from different pages.
My question is if there is a way to avoid pulling in duplicated questions by filtering the question field. I tried to use a workaround by using the question as entity ID but I wasn’t satisfied with the result as this way the connector wasn’t pulling all the FAQ and I was missing ~60 FAQs with no clear reason).
Have you ever had a similar use case?

Thanks!

Fede

2

Hi @Federica_Carrus ,

There is not currently an automatic way of doing this. We would recommend manually deleting duplicates. This would be a great feature to post in the idea board for our product managers to review!

Thanks,
Melissa

1 Like
3

Does this still stand?

4

Hi @Amanda_Mackler - as of now, yes. We’d highly recommend posting on Ideas about this to surface the use case to the product team as well as other users who may have the same need!