RapidMiner Data Science Competition 4: DrivenData's "Pover-T" - $15,000 prize for charity

sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
edited November 2018 in Help

hello RapidMiners - 

 

Greetings and happy new year. First of all huge congratulations again to @maros_plsik for winning the Fantasy Football challenge. And kudos to @florian_ziegler and @yzan for their great efforts as well. Unfortunately those were the ONLY submissions to this challenge so I'm going to change gears for the next one in hopes that we can get more people involved.

 

The_World_Bank_logo.svg.pngSetup: DrivenData is a data science competition website (similar to Kaggle) where they only host challenges that benefit the public good. They have a new challenge called "Pover-T", sponsored by the World Bank, where they are trying to use survey data to predict which households are classified as "poor" or not. It's pretty straightforward. The measuring stick is simply minimizing logarithmic loss on the "poor" class predictions. They have standard training and testing data sets, a scoreboard, and so on.  Here's the link where you can read all about it: https://www.drivendata.org/competitions/50/worldbank-poverty-prediction/page/97/

 

Challenge: I have formed a "RapidMiners" team on DrivenData where I am inviting ANY and ALL RapidMiner data scientists who wish to join me. If we win (I have no doubt we will!), we all agree to donate the full $15,000 prize money to Doctors Without Borders / Médecins Sans Frontières. Why? First because I think it's a very good cause. Second because splitting prize money among n RapidMiners in k countries would be horribly difficult. Third because I want to show those folks at SAS that they are not the only ones doing #data4good (look it up on Twitter).

 

RapidMiner processes: I worked on it for a couple of hours last night and we're already in 29th place. Let's keep trading processes in this thread and go from there. I have a folder on OneDrive that I am sharing here; probably the "right" way to do this is on GitHub but I'm a git-idiot. Anyone want to get this going? 

 

Deadline: The "Pover-T" challenge is going on now and ends Wednesday, February 28, 2018, 11:59 p.m. UTC.

 

Who's in? Sign up at DrivenData, send me your username, and let's show the world our data science chops!

 

Scott

 

(image source: Wikimedia Commons)

 

 

Answers

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Count me in!  I'm Telcontar120 at DrivenData as well if you want to add me to the team.  

     

    Another option for sharing work would be to use the RapidMiner competition server?  We could create a shared repository where we keep the data files, and then version off the different processes. @sgenzer if you could set something up like that it would definitely be a lot more efficient than swapping files on OneDrive or github.  

     

     

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    count me in too, i'm at: neuralmarket

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    awesome. Thanks @Thomas_Ott @Telcontar120. You need to sign up for the competition first and then I can add you as teammates. @mschmitz is, of course, already on the ball with a score of around 0.2 after 30 min. :)

     

    Good idea about the competition server. It's still running. Ping me if you need access.  Note that the server is running 7.6 so you need to run Studio 7.6 to access. :(

     

    Let's DO this!

     

    Scott

     

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Any chance to update the competition server to version 8?  I no longer have a running version of Studio 7.6 and since they can't co-exist on the same machine there is no easy way to get one going again...

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    Competition Server should be set up

     

    Screen Shot 2018-01-03 at 9.19.22 AM.png

     

    Scott

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    yeah you're right - we need to upgrade that server. OK I will ping the folks in Budapest.

     

    Scott

     

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    meanwhile here are my first two processes - the second is still me playing around with ETL of the "indiv" data set in Company A.

     

    Scott

     

     

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Looks like I need to finally upgrade to V8 then. Perfect timing, I need to put a V8 server on a new box.

  • DocMusherDocMusher Member Posts: 333 Unicorn

    Hi my RM friends,

    would love to join this competition

    hopefully I can be of any added value...

    Cheers

    Sven

    https://www.drivendata.org/users/SvenVanPoucke/

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    that's great, @SvenVanPoucke! Sign up for the competition first and then I can add you to the team.

     

    Scott

     

  • PseudoCyantistPseudoCyantist Member Posts: 1 Contributor I
    Hello friend
    U can count me in,I have never tried drivendata before but I have good experience with kaggle.I have register just now.My username is crazy_rules.I have Good experience with weka,Rstudio,Rapidminer,Anaconda(keras installed).I know feature selection,feature engineering and have a Good knowledge on performance of various machine learning algorithms.

    Thank you in advance.
  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Looks like this is still running v 7.6 @sgenzer?  I can't get in with v8 this morning.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    @PseudoCyantist - great to have you aboard. Just sent you an invite.

     

    @Telcontar120 yes still on 7.6. Very sorry about the delay. I will push again today.

     

    Scott

     

  • AndrewAndrew RapidMiner Certified Expert, RapidMiner Certified Master, Member Posts: 47 Guru

    Hello all

     

    I fancy trying this - I've been wanting a reason to play with Keras. I'm andrewchisholm on DrivenData.

     

    Andrew

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    great @Andrew - you need to sign up for the competition first, then I can add you to the team. And if anyone needs access to the RapidMiner Competition Server, let me know (yes @Thomas_Ott it's still RM 7.6...working on it!).

     

    Let's DO this!


    Scott

     

  • AndrewAndrew RapidMiner Certified Expert, RapidMiner Certified Master, Member Posts: 47 Guru

    Hello @scott, I joined the competition.

  • DocMusherDocMusher Member Posts: 333 Unicorn

    Hi,

    Any news on RM Server 8 for this competition?

    :smileyhappy:

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    Hi - so the good news is that I have the ear of the person who can move the server to RM8. Bad news is that it cannot happen until next week. Thanks for your patience!

     

    Scott

     

  • curious95curious95 Member Posts: 12 Contributor II

    I'll be happy to work for the cause. Count me in !!

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    that's great. Thank you! Please 1) sign up with DrivenData, 2) join the Pover-T competition, and 3) send me your username. I will add you to the team.


    Scott

     

  • JEdwardJEdward RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 Unicorn

    @sgenzer  My username is JEdward.  Add me in when you get a chance.

     

     

  • ranindrastiaranindrastia Member Posts: 1 Contributor I

    @sgenzer I already sign up the DrivenData and sign up the competition.

    Please kindly add me to the team, my username is ranindrastia

    Thank you.

     

    Regards,

    Rani

    http://www.rapidminerchina.com/

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    hello @JEdward and @ranindrastia - thank you for joining us! Please let me know if you need access to the RapidMiner competition server as this is where (ideally?) we will share code.

     

    FYI @mschmitz has given me his latest effort that I will post this week. Hopefully our rankings will rise! I will also post his work on the server.

     

    Scott

     

  • DocMusherDocMusher Member Posts: 333 Unicorn

    Hi,

    Could someone send me a PM with access to the server for this competition?

    Sven

Sign In or Register to comment.