RapidMiner 9.7 is Now Available

Lots of amazing new improvements including true version control! Learn more about what's new here.

CLICK HERE TO DOWNLOAD

COVID-19 board

sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,952  Community Manager
edited March 20 in Help
Hello all RapidMiner community members -

I have been hesitant to post a new discussion about COVID-19 as there is so much already out there, but I am sincerely concerned about the well-being of our RapidMiner family. I am also very interested to hear if anyone out there is either (a) working on any data science COVID-19 projects, and/or (b) any service projects that you may be leading/participating in that is helping COVID-19 patients or research in your local community.

So please use this discussion board to share, discuss, and support one another. I sincerely hope you are well during this very difficult time, and my deepest sympathies to those who are either ill or directly affected by friends or family that are suffering.

Scott
Tagged:
Jasmine_varunm1mharmsyyhuangOscare369mlubiczrfuentealbaHasnain
«1

Comments

  • RaedMarjiRaedMarji Member Posts: 13 Contributor II
    for anyone who wants to help Kaggle is having a special competition for Covid-19. not sure if you can use RapidMiner. but any input there would be helpful I'm sure.
    https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge/
    [Deleted User]Pavithra_Raodedeersgenzer
  • cs_sneedcs_sneed Member Posts: 2 Contributor I
    I work for a logistics consulting firm that has a contract with DLA. We just completed a study of the domestic capacity for reagents. The study concluded that normal capacity was not at risk for the continental U.S. Surge, meaning the ability to produce large quantities quickly, was not studied and now we are being questioned on our conclusions. Keeping in mind that capacity and surge are two different things, we are continuing to study data trends along the supply chain and provide answers to our client, DLA. That said, the level of data science involved from our specific firm has been minimal. Many other government organizations are providing superb graphics and models that we are passing along. Our primary concern is how the disrupted supply chains ultimately affect the U.S. warfighter. Thus, it's not the reagents or the related medical equipment that is of concern. It's how the shutdown and quarantine protocols affect production and delivery for all of the OTHER supply chains critical to readiness. What data sets are others looking at for how the shutdown is affecting logistics?
    varunm1DocMushersgenzer
  • User23311User23311 Member Posts: 4 Contributor I
    Dear all,

    DocMusher : your post is impressive, thanks for sharing!

    My wife and me both work for big companies having global production of goods for multiple customers. Both companies still handle the topic mostly manual, consolidating data from several stakeholders manually in order to roughly identify potential countermeasures, aligning with governments or re-focus strategies.
    Why?  
    Data science needs well designed models and structures. You can perform data analysis from SAP easily, but what are the answer you will find and what are measures/actions you derive from the analysis? Countermeasures/actions in regard to this unforeseen situation still needs human intelligence, creativity and flexibility, which should be available in good management (but not mandatorily IS available ;-))...

    ************

    I started to collaborate with @mbs (see other post: https://community.rapidminer.com/discussion/56951/huge-field-trial-regarding-global-economy-ecology-and-society#latest). 

    We want to analyse global data in multiple areas in order to evaluate positive and negative effects of covid19 on ecology, economy and society. As an outcome, my wish is to publish a paper with results that are beneficial for the society (e.g. key messages easy to read and having impact on daily live...)


    We came to following interrim conclusion:

    1. We have to wait until more data are available, minimum 150days.
    --> However, some data are only available on daily basis, so we are collecting some single data separately

    2. We have to ask the right questions in order to find the right answers (sounds simple, but is essential).
    --> What is possible to derive from the data, what is logical?

    3. It is difficult to get data from different sources. 
    --> I am willing to get access to statista or tradingeconomics, but it is also helpful to check other sources or find partners to deliver information such as information on consumption, surveys, newspaper articles etc. But i can't spend too much time for dilligence work...

    4. And maybe the biggest challenge is to differ between "natural fluctuations of economy", direct effects of COVID19 and indirect effects, thus classical question of causality...
    --> Therefore I think on having sub-model approach, thus dividing the overall topic into sub-systems with different topics. And adding also fixed relations (Y=a*X) and logical relations (if 'A' and 'B', then 'C')

    Who else is willing to collaborate within this activity? 
    Minimum outcome is personal learning on how to deal with such kind of complex data analysis problems. Maximum outcome is a paper that might have impact to our society.

    Please give PN and let's align on HOW to collaborate!

    Jan
    [Deleted User]DocMushervarunm1sgenzer
  • DocMusherDocMusher Member Posts: 329   Unicorn
    edited March 22
    I am always interested in any initiative with impact for now or for a next wave, but enabling flows, data gathering, predictive analysis, etc really should result in a measurable impact, otherwise its only interesting. I am in when ROI (valued by " reducing damage control") is the primary goal. 
    Thanks
    Sven
    sgenzer
  • User23311User23311 Member Posts: 4 Contributor I
    Sven, i fully understand your comment. You are "hands-on" and focussed on COVID19.
    However, ROI can be short-term and selective or long-term and holistic.
    I am Scientist and I want to change long-term. Based on this single disruptive impact of COVID, I want to find understanding, clear rational and cause-effect-relation evidence EXAMPLES, how we as human being shall change our behavior in order to act sustainable for the future. Furthermore, I want to find ways, how to derive valuable information from data - the essential task of data science (knowing that there are plenty of theories already available). 
    For concrete action, there are plenty of tools such as this paper showing how to predict COVID spread via Kalman-Filter: https://towardsdatascience.com/using-kalman-filter-to-predict-corona-virus-spread-72d91b74cc8

    [Deleted User]sgenzer
  • DocMusherDocMusher Member Posts: 329   Unicorn
    Fully agree, keep me in touch, contact me where I can help, I am in.
    Sven
    sgenzer
  • [Deleted User][Deleted User] Posts: 0 Learner III
    @DocMusher   and  @User23311,

    Hello 

    Great discussion  :)

    I agree with you. Let continue it with private message. 

    Thank you
    mbs
    sgenzer
  • varunm1varunm1 Moderator, Member Posts: 1,203   Unicorn
    @User23311

    Very interesting and useful. If you need anything related to Data science, do let me know. I am more than happy to help with this project.
    Regards,
    Varun
    https://www.varunmandalapu.com/

    Be Safe. Follow precautions and Maintain Social Distancing

    sgenzer
  • [Deleted User][Deleted User] Posts: 0 Learner III
    edited March 22
    @DocMusher , thank you for the link.( very useful)

    sgenzer
  • User23311User23311 Member Posts: 4 Contributor I
    @DocMusher 
    Well, this is not big data, but giant data...
    This is way above our level regarding power.
    But I was wondering what you can do with more data available for simple folks like us. Public available. And combining those diverse and meaningful data to new insights/findings.
    [Deleted User]sgenzer
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,509  RM Data Scientist
    Hey @dedeer ,
    nice work. I saw on your profile, that you search for a twitter connector for sentiment analysis. Is this solved?

    BR,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
    [Deleted User]
  • DocMusherDocMusher Member Posts: 329   Unicorn
    @dedeer
    Indeed nice work, would you mind to share your process here?
    If someone is interested in a general epidemic calculator, this looked interesting although I lack the time to review in depth, I think @mschmitz has the background to assess its correctness from math point of view. 
    Cheers, stay healthy, use this period as an opportunity to value what is important.
    Sven
    sgenzerdedeer
  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,952  Community Manager
    hi all -

    I am just amazed and awed by this discussion. I think I speak for everyone at RapidMiner saying that you are all in our thoughts. Huge respect for @DocMusher for his amazing work and powerful message.

    Scott

  • dedeerdedeer Member Posts: 9 Contributor II
    Hey @mschmitz
    Thank you very much for your follow up. I am very new in the community and it is great that RM has a very powerful one. 
    I figured out a way with Twitter and Aylien (not supported anymore I guess) connections and Aylien Extract Sentiment Block for quick look-up the re-tweets sentiment (objective/subjective) after health minister's tweet + 2 hours. 

    I just want to add text processing by creating a document of the most recent 100 tweets every hour updated and see some words' vectors during this period about Covid-19. 

    Any suggestions for improving? 

    Thanks all, stay healthy. 


    [Deleted User]
  • dedeerdedeer Member Posts: 9 Contributor II
    @DocMusher
    Thank you very much for the link it is very great for the application of math models. I think I can inject them in to Generate Attribute block and create a forecasted set for each country and/or state. 
    Here is my design view for ETL. 

    You can append and run for playing with the dataset at the Results View Visualization tool. I made a filter for China and Confirmed cases then selected attributes for dates and date-related attributes, applied transpose. Output as below: 

    Now we can easily visualize rate like: 

    From the first day of the confirmed case till the last date. As w can observe there is a good resistance at some point which is correlated with the population at the state, that confirmed cases begin decelerating. We can add text data of related country's precautions by authorities, plus or minus couple days, we can measure how it worked. 

    It is very premature at the moment. I have another design with a neural network block to estimate the next days' confirmed cases. 

    Any suggestions? 
    Stay healthy.



    [Deleted User]Pavithra_Rao
  • DocMusherDocMusher Member Posts: 329   Unicorn
  • dedeerdedeer Member Posts: 9 Contributor II
    @DocMusher
    it is excellent! thanks for sharing. 
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,509  RM Data Scientist
    Hi @dedeer,

    I just want to add text processing by creating a document of the most recent 100 tweets every hour updated and see some words' vectors during this period about Covid-19. 
    You need a RapidMiner Server for this, which is part of the educational program. Do you have one?

    Best,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
    sgenzer
  • dedeerdedeer Member Posts: 9 Contributor II
    @mschmitz
    Unfortunately no, I don't have RM Server. I didn’t know I could reach one. 
    Best,
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,509  RM Data Scientist
    RM Server can be installed on your laptop. THen of course your laptop needs to run to execute the process every day. You can find installation guides and so on at: https://docs.rapidminer.com/latest/server/

    Best,
    Martin

    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
    dedeer
  • DocMusherDocMusher Member Posts: 329   Unicorn
  • DocMusherDocMusher Member Posts: 329   Unicorn
    I recently received this answer to my social distancing question. The critical issue here is the balance of individual location privacy vs. public health common good. The latter takes precedence over the former during major public health emergencies and pandemics. In Israel, for example, they had to invoke emergency spy powers (https://www.bbc.co.uk/news/technology-51930681) to develop their COVID-19 HaMagen app (https://www.standardmedia.co.ke/article/2001365512/israel-launches-app-alerting-users-of-exposure-to-coronavirus). In Canada, Toronto is using cellphone location data from telecoms to encourage social distancing (https://nationalpost.com/technology/city-of-toronto-gathering-cellphone-location-data-from-telecoms-in-bid-to-slow-spread-of-covid-19-tory/wcm/f916f892-b47d-43a4-85aa-2a214e136ee0). Other countries are doing the same/developing similar apps; see some relevant news links at http://healthcybermap.org/WHO_COVID19/#10
    P.S. A description of the corresponding app in use in China is available at https://tinyurl.com/wrccsfw (video: https://youtu.be/3K3fy5eKeuM?t=722)
    sgenzerdedeer
  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,952  Community Manager
    wow @DocMusher I had not seen these apps before. Thank you. Fascinating.

    On a similar note, I was reflecting this morning that this COVID-19 pandemic will provide data for at least 20 years of PhD theses in every field imaginable...
    DocMusherdedeerhbajpai
  • PiciaPicia Member Posts: 10 Contributor I
    I would appreciate if people from different countries could write either directly to me or put posts whether in their country the tuberculosis vaccine is obligatory in their country and if not, in which year it ceased to be obligatory. There is a hypothesis, that this vaccination alleviates the course of illness.
  • DocMusherDocMusher Member Posts: 329   Unicorn
Sign In or Register to comment.