python调用sparksql 使用Python从MySQL数据库表读取SparkSQL

I have a 'user' table in MySQL. I want to read it to my Spark SQL program. How can I read the table from MySQL to the Apache Spark's SparkSQL module using Python? Is there a connector I can use for this task?

Thanks.

解决方案

There is a similar question answered. Start pyspark like this

./bin/pyspark --packages mysql:mysql-connector-java:5.1.38

Then just run

sqlContext.read.format("jdbc").options(

url ="jdbc:mysql://localhost/mysql",

driver="com.mysql.jdbc.Driver",

dbtable="user",

user="root",

password=""

).load().take(10)

This would most likely just work. But this depends on your mysql set-up, so if it doesn't try changing password, username, db-url and other settings.