使用pyspark时报错 您所在的位置:网站首页 pyspark使用map报错 使用pyspark时报错

使用pyspark时报错

2024-07-11 11:02| 来源: 网络整理| 查看: 265

使用pyspark时报错? 所有的都调试好了,cmd运行pyspark也可以,但在python里的ide运行这一段程序总是报错。

from pyspark import SparkConf, SparkContext

from py4j.java_gateway import JavaGateway

# 初始化Spark conf = SparkConf().setMaster("local").setAppName("My App") sc = SparkContext(conf=conf) print("-----------------2-------------------") lines = sc.textFile("E:///JinXiejie/spark-2.2.0-bin-hadoop2.7/README.md") pythonLines = lines.filter(lambda line : "Python" in line) print("---------------3-----------------") print pythonLines.first()

报告错误类型: C:\Python34\python.exe E:/JinXiejie/PythonCases/PyDemo/Pydemo.py

-----------------1-------------------

Traceback (most recent call last):

File "E:/JinXiejie/PythonCases/PyDemo/Pydemo.py", line 10, in

sc = SparkContext(conf=conf)

File "C:\Python34\lib\site-packages\pyspark\context.py", line 115, in __init__

SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)

File "C:\Python34\lib\site-packages\pyspark\context.py", line 283, in _ensure_initialized

SparkContext._gateway = gateway or launch_gateway(conf)

File "C:\Python34\lib\site-packages\pyspark\java_gateway.py", line 80, in launch_gateway

proc = Popen(command, stdin=PIPE, env=env)

File "C:\Python34\lib\subprocess.py", line 859, in __init__

restore_signals, start_new_session)

File "C:\Python34\lib\subprocess.py", line 1112, in _execute_child

startupinfo)

FileNotFoundError: [WinError 2] 系统找不到指定的文件。

Process finished with exit code 1

解决方案: 打开PyCharm,run->edit-Configurations->Environment variables

分别添加PYTHONPATH和SPARK_HOME如下(具体路径为自己电脑spark文件位置):



【本文地址】

公司简介

联系我们

今日新闻

    推荐新闻

    专题文章
      CopyRight 2018-2019 实验室设备网 版权所有