reading Facebook open graph into database

robinrobin Member Posts: 100 Guru
edited November 2019 in Help

Hi

 

I am wantingto read Facebook opengraph information into a clients DB. I have formatted the input into Facebook and I am processing the recursive requests. The problem I am having is that I am just not able to get the JSON into a format that I can read into the DB. Multple splits and transposes later and I am still no where. Is there an easier way to read this format of code in: 

 

 

{  
   "posts":{  
      "data":[  
         {  
            "comments":{  
               "data":[  
                  {  
                     "created_time":"2018-04-19T06:09:10+0000",
                     "from":{  
                        "name":"User 1",
                        "id":"12345"
                     },
                     "message":"Thank you",
                     "id":"12345_12345"
                  },
                  {  
                     "created_time":"2018-04-19T06:11:50+0000",
                     "from":{  
                        "name":"User 2",
                        "id":"567890"
                     },
                     "message":"for your help",
                     "id":"12345_567890"
                  },
               ],
            }
         }
      ]
   }
}

The format it comes from in the open graph means that there can be sub comments of comments which is where I am fowling up as I end up with a matrix instead of a data table. 

 

Tagged:

Best Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn
    Solution Accepted

    @SGolbert and @robin one other possibility is that FB is changing their API's because of this whole Cambridge Analytica business. 

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Solution Accepted

    so yes @robin I too am having trouble with your JSON array. Something weird about its formatting.

     

    FYI the "JSON to Data" operator in the Text Processing extension really likes everything to be in {} rather than []. A simple "Replace Tokens" operator is usually my fix for this.

     

    Scott

     

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    @robin I'm getting a malformed JSON file, what kind of error are you getting?

  • SGolbertSGolbert RapidMiner Certified Analyst, Member Posts: 344 Unicorn

    Hi Robin,

     

    I am also facing some troubles with the Facebook operators. Can you post a sample process? I've been able to get some data using the Python SDK, but I would like to use RapidMiner instead.

     

    Going back to your post, maybe you could try generating an attribute "parent comment id", leaving it blank for an initial comment. Note that you will be losing some information, because the subcomment will no longer be chained (maybe they can be sorted by time).

     

    Looking forward to seeing a process!

     

     

  • SGolbertSGolbert RapidMiner Certified Analyst, Member Posts: 344 Unicorn

    I think that the search capabilities are gone from the Graph API. I am able to obtain information, but I must provide some user id to begin the search.

     

    The documentation on the API is also quite bad.

Sign In or Register to comment.