数据科学家的8类数据分析工具推荐

最近,北卡来罗纳大学的一位数据科学家Jefferson Heard分享了自己多年来收集沉淀的数据分析工具集,希望对每一位数据科学家有用:

1、处理较大、较复杂的类excel数据

  • Pandas -处理tabular(类似Excel)数据的通用工具套件
  • SQLite – Tabular数据库格式,能够处理大规模数据集,同时也能在桌面环境运行。
  • PostgreSQL – 企业级数据库系统

2、处理空间、地理数据:

  • PostGIS – Postgres的地理空间数据类型扩展
  • Carto – 地理空间数据的商业数据挖掘工具
  • Mapbox – 商业地图绘制工具,同时也是一个web地图系统。
  • Leaflet – 基于网络资源和本地数据开发活动web地图的代码库
  • qGIS – 适用于几乎所有地理空间和地图绘制的图形化GIS工具

3、处理非常规数据:

  • RethinkDB – 处理实时数据流非常棒的数据库,正在从商业转开源,小心使用。
  • MongoDB – 处理大规模非结构化和半结构化数据的流行数据库,应用于生产环境需要加小心。
  • CouchDB – 与MongoDB有些类似但不尽相同。
  • Cassandra – 图谱和关系数据库

4、为大规模数据集创建性能代码:

  • Pandas – Python下一个开源数据分析的库,它提供的数据结构DataFrame极大的简化了数据分析过程中一些繁琐操作。
  • Apache Spark – 一个通用的高性能数据处理系统
  • SciPy and Numpy -可编写脚本的基于C的数值算法,能在紧凑的,底层机器数据架构上运行。
  • Cython – 使用用C编译器的Python编译器,用来提升Python性能。
  • PyOpenCL – 在图形显卡上进行数值计算和统计处理。

5、数据清洗工具

  • ODO – 在不同数据格式间进行转换的Python库。
  • OpenRefine – 拥有图形用户界面的数据发现和清洗工具
  • Pandas – 数据科学任务中用来处理tabular数据的通用Python工具集
  • Scrapy – Python开发的一个快速,高层次的屏幕抓取和web抓取框架,用于抓取web站点并从页面中提取结构化的数据。
  • BeautifulSoup – 与Scrapy类似但不尽相同
  • Scrubadub – 去除个人身份信息
  • Arrow – 帮你轻松驾驭日期和时间戳的Python库
  • DataCleaner – 剔除脏数据的Python库
  • Dora – 与DataCleaner功能类似的Python库。

6、数据可视化工具

  • Processing – 交互式开发交互式可视化内容. 推荐读本:Visualizing Data
  • D3 – 在web上开发可视化交互
  • C3 – 来自D3的图表.
  • Bokeh – 与D3类似, 但基于Python.
  • matplotlib – 最早的Python数据可视化工具集。
  • Leaflet – 一个为开发移动设备友好的互动地图的开源 JavaScript 库。
  • MapBox -详见地图工具集。
  • qGIS – 详见地图工具集。
  • VTK – 在医疗、  和物理研究领域常用的重型可视化工具包。

7、数据挖掘和机器学习工具

  • Weka – 一个机器学习和数据挖掘工具包,这里有一本免费可读的参考书
  • SciKitLearn – 基于Python的机器学习和数据挖掘工具套件。
  • Orange – 另一个基于Python的数据挖掘工具套件,同样拥有图形用户界面。
  • TensorFlow – Google开源的多维度图谱数学建模工具。

8、分享、协作以及知识管理工具

  • Django -基于Python的web框架
  • Django REST Framework – 为Django网站创建 REST APIs
  • IRODS – 企业级数据存储和管理,包括元数据管理和基于规则的数据处理。
  • Cassandra (useful for metadata and relationship storage) – 一个存储和查询元数据经常用到的开源分布式数据管理系统
  • GitLab -GitHub的开源替代品,可搭建私人服务器。
  • ReciPy –
  • Prov – Python implementation of the W3C provenance model
  • Kanren (部署基于元数据和数据源信息的业务逻辑非常有用) – 一个描述性Python逻辑编程系统,非常适合科学元数据的查询和基于规则的处理。

未经允许不得转载:守心斋 » 数据科学家的8类数据分析工具推荐

赞 (0)

相关推荐

    暂无内容!

评论 829

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
  1. Destination WeddingsThankfulness to my father who shared with me about this website, this website is genuinely amazing.回复
  2. new york spammerAn impressive share! I've just forwarded this onto a friend who has been doing a little research on this. And he in fact bought me breakfast because I discovered it for him... lol. So allow me to reword this.... Thank YOU for the meal!! But yeah, thanx for spending some time to talk about this subject here on your site.回复
  3. M88Wow! This blog looks exactly like my old one! It's on a completely different topic but it has pretty much the same layout and design. Great choice of colors!回复
  4. dxracer reviewor fashionable and useful, you will easily find your Id Nike Blazers sandals at a discount price to fit your budget.回复
  5. télécharger album maitre gims ceinture noireThat's why they see no issue in staying unproductive or even with stealing from you. You can use a free of charge telephone tracker app but they are really effortless to detect and do not do close to as very much as this app does. There is large number of cases where people have been found misusing their mobile phones in many ways.回复
  6. visit websiteTotally agree with you, about a week ago wrote about the same in my blog..!回复
  7. suco verde rico em ferroA aprovação destinado a adolescentes acima de 2 anos no entanto adolescentes também 20 anos no entanto a mesma da anosidade mais 5 gramas por tempo.回复
  8. télécharger maitre gims ceinture noireIf you are purchasing a new phone for yourself, your teen, or your significant other, seek out a GPS-enabled model. In order not to sound like that, you need a voice changer that has more than 2 or 3 adjustments. The one drawback to this though, is that often people who text each other a lot develop their own shorthand that only the two of them understand.回复
  9. for detailsWhat as up to every body, it as my first pay a visit of this web site; this website consists of amazing and genuinely good data designed for visitors.回复
  10. Agen Togel TerpercayaI am really enjoying the theme/design of your web site. Do you ever run into any web browser compatibility problems? A number of my blog readers have complained about my website not operating correctly in Explorer but looks great in Opera. Do you have any solutions to help fix this problem?回复
  11. technology news updatesThank you for another wonderful post. Where else could anyone get that kind of information in such a perfect method of writing? I've a presentation next week, and I am at the look for such information.回复
  12. karrueche tranVery nice post. I just stumbled upon your blog and wanted to say that I've really enjoyed surfing around your blog posts. After all I'll be subscribing to your feed and I hope you write again very soon!回复
  13. hunting bowsIf you are going for most excellent contents like回复
  14. Home ImprovementI truly enjoy reading through on this internet site, it has wonderful blog posts. "And all the winds go sighing, For sweet things dying." by Christina Georgina Rossetti.回复
  15. ارقام توكيل توشيبا البحيرةJust a smiling visitant here to share the love (:, btw great style and design.回复
  16. NFL SCHEDULE 2018 TEAM BY TEAM,Valuable information. Lucky me I discovered your web site by chance, and I am stunned why this coincidence did not came about earlier! I bookmarked it.回复
  17. EPFO Login PortalHi there! I know this is kind of off topic but I was wondering which blog platform are you using for this website? I'm getting tired of Wordpress because I've had problems with hackers and I'm looking at alternatives for another platform. I would be awesome if you could point me in the direction of a good platform. To Know More About UAN Member Portal , Please Visit Here: http://www.epf-uanlogin.co.in回复
  18. krutu didinimasIš tikrųjų nesvarbu, ar kažkas ne žinoti po to jos iki kito vartotojai, jie bus padėti, ir čia atsitinka.回复
  19. dentysta gorzowSweet blog! I found it while browsing on Yahoo News. Do you have any tips on how to get listed in Yahoo News? I've been trying for a while but I never seem to get there! Many thanks回复
  20. http://www.koinonia.orgIt's very trouble-free to find out any matter on net as compared to books, as I found this piece of writing at this web page.回复
  21. keluaran sidneyWoah! I'm really enjoying the template/theme of this website. It's simple, yet effective. A lot of times it's very hard to get that "perfect balance" between superb usability and visual appeal. I must say that you've done a fantastic job with this. Also, the blog loads extremely quick for me on Safari. Superb Blog!回复
  22. Home ImprovementPretty section of content. I just stumbled upon your site and in accession capital to assert that I get in fact enjoyed account your blog posts. Any way I’ll be subscribing to your feeds and even I achievement you access consistently fast.回复
  23. di SubulussalamGreetings! Very helpful advice in this particular article! It is the little changes that make the biggest changes. Many thanks for sharing!回复
  24. technology news updatesIts like you read my mind! You seem to know a lot about this, like you wrote the book in it or something. I think that you could do with a few pics to drive the message home a little bit, but instead of that, this is magnificent blog. An excellent read. I will certainly be back.回复
  25. funnyIts like you read my mind! You seem to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a bit, but instead of that, this is great blog. A fantastic read. I'll definitely be back.回复
  26. Technology Current EventsThanks , I have recently been looking for information about this subject for a while and yours is the best I've found out so far. However, what concerning the bottom line? Are you sure in regards to the source?回复
  27. fuckHey very nice blog!回复
  28. aimbot cs goEngineers have come up with a solution to resolve these issues with a help of mobile software that will act as a mobile spy to monitor all the activities in a particular mobile phone. They offer features like Two Wash Courses (Gentle & Normal Wash, twin water inlets, spin shower, two wash courses (Gentle and Normal wash), and wheels for easy mobility in some models. Do you suspect that your employee is performing something mistaken with your enterprise.回复
  29. polytronglassAsking questions are genuinely nice thing if you are not understanding something entirely, except this paragraph offers good understanding yet.回复