我正在嘗試將數據從 Google BigQuery 加載到在 Google Dataproc 上運行的 Spark(我正在使用 Java)。我嘗試按照此處的說明進行操作:https ://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example我收到錯誤:“ ClassNotFoundException: Failed to find data source: bigquery。”我的 pom.xml 如下所示:<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.virtualpairprogrammers</groupId> <artifactId>learningSpark</artifactId> <version>0.0.3-SNAPSHOT</version> <packaging>jar</packaging> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.11</artifactId> <version>2.3.2</version> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-sql_2.11</artifactId> <version>2.3.2</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-hdfs</artifactId> <version>2.2.0</version> </dependency> <dependency> <groupId>com.google.cloud.spark</groupId> <artifactId>spark-bigquery_2.11</artifactId> <version>0.9.1-beta</version> <classifier>shaded</classifier> </dependency> </dependencies>將依賴項添加到我的 pom.xml 后,它會下載大量內容來構建 .jar,所以我認為我應該擁有正確的依賴項?然而,Eclipse 還警告我“從未使用過 import com.google.cloud.spark.bigquery”。
ClassNotFoundException:找不到數據源:bigquery
慕運維8079593
2024-01-25 21:29:53