I have scraped a lot of articles from the internet in a PostgreSql database, where the variable with the article is named Text. I also have a second dataset from Excel with a variable there have names of danish persons.
My problem is that I can't figure out a calculated field there can search the Text variable through and see how many times different danish persons from the other dataset is found.
Any good ideas. Thank you.
I believe you can accomplish this with cross database joins now. I don't have a Postgre database to test explicitly with, but the option did appear when I went to set up another connection in the same data source.
Steps:
1) connect to excel/csv file with names
2) on data source, click Add button next to connections
3) bring in your postegre dataset
4) on joining fields, click empty cell and select create join calculation
when prompted, enter 1 as value (or any value at all, this will assign that value to every record in your dataset), do the same on the right side, using the same value.
5) now you have a dataset where every document is related to every name, and you can check row by row if the name is in the text. My calculation looks like this:
A few things to be wary of:
1) this join is a cross join, creating a cartesian dataset - the size of the dataset is #records in first table * #records in second table, which may become very, very large if you have two large datasets
2) please take note of the sample texts I put in - using the simple CONTAINS method, there are some gotcha's in terms of the one text contained within another.
Let me know how this looks.
