This is the Scala version of the approximation algorithm for the knapsack problem using Apache Spark. I ran this on a local setup, so it may require modification if you are using something like a Databricks environment. Also you will likely need to setup your Scala environment. All the code for this is at GitHub First, let's import all the libraries we need. import org.apache.spark._ import org.apache.spark.rdd.RDD import org.apache.spark.SparkConf import org.apache.spark.SparkContext._ import org.apache.spark.sql.DataFrame import org.apache.spark.sql.SparkSession import org.apache.spark.sql.functions.sum We'll define this object knapsack, although it could be more specific for what this is doing, it's good enough for this simple test. object knapsack { Again, we'll define the knapsack approximation algorithm, expecting a dataframe with the profits and weights, as well as W, a total weight. def knapsackApprox(knapsackDF: DataFrame, W: Double): Da
Technical notes about past publications and work by Darrell Ulm including Apache Spark, software development work, computer programming, Parallel Computing, Algorithms, Koha, and Drupal. Source code snippets, like in Python for Spark. Retrospective of projects.