| --master |
MASTER_URL |
spark://host:port, mesos://host:port, yarn, or local |
| --deploy-mode |
DEPLOY_MODE |
client/cluster |
| --class |
Your application's main class |
|
| --name |
A name of your application |
|
| --jars |
Comma-separated list of local jars to include on the driver and executor classpaths |
以逗号分隔 |
| --packages |
Comma-separated list of maven coordinates of jars to include on the driver and executor classpaths |
The format for the coordinates should be groupId:artifactId:version. |
| --exclude-packages |
Comma-separated list of groupId:artifactId, to exclude while resolving the dependencies provided in --packages to avoid dependency conflicts. |
|
| --repositories |
Comma-separated list of additional remote repositories to search for the maven coordinates given with --packages. |
|
| --py-files |
Comma-separated list of .zip, .egg, or .py files to place on the PYTHONPATH for Python apps. |
|
| --files |
Comma-separated list of files to be placed in the working directory of each executor. |
|
| --conf |
Arbitrary Spark configuration property. |
|
| --properties-file |
Path to a file from which to load extra properties. If not specified, this will look for conf/spark-defaults.conf. |
|
| --driver-memory |
Memory for driver (e.g. 1000M, 2G) |
|
| --driver-java-options |
Extra Java options to pass to the driver. |
|
| --driver-library-path |
Extra library path entries to pass to the driver. |
|
| --driver-class-path |
Extra class path entries to pass to the driver. Note that jars added with --jars are automatically included in the classpath. |
|
| --executor-memory |
Memory per executor (e.g. 1000M, 2G) |
Default: 1G |
| --proxy-user |
User to impersonate when submitting the application. This argument does not work with --principal / --keytab. |
|
| --help |
Show this help message and exit. |
|
| --verbose |
Print additional debug output. |
|
| --version |
Print the version of current Spark. |
|
| --driver-cores |
Cores for driver (Default: 1). |
Spark standalone with cluster deploy mode only |
| --supervise |
If given, restarts the driver on failure. |
Spark standalone or Mesos with cluster deploy mode only |
| --kill |
If given, kills the driver specified. |
Spark standalone or Mesos with cluster deploy mode only |
| --status |
If given, requests the status of the driver specified. |
Spark standalone or Mesos with cluster deploy mode only |
| --total-executor-cores |
Total cores for all executors. |
Spark standalone and Mesos only |
| --executor-cores |
Number of cores per executor. (Default: 1 in YARN mode, or all available cores on the worker in standalone mode) |
Spark standalone and YARN only |
| --driver-cores |
Number of cores used by the driver, only in cluster mode(Default: 1). |
YARN-only |
| --queue |
The YARN queue to submit to (Default: "default"). |
YARN-only |
| --num-executors |
Number of executors to launch (Default: 2). |
YARN-only |
| --archives |
Comma separated list of archives to be extracted into the working directory of each executor. |
YARN-only |
| --principal |
Principal to be used to login to KDC, while running on secure HDFS. |
YARN-only |
| --keytab |
The full path to the file that contains the keytab for the principal specified above. |
YARN-only |
网友评论