Hadoop with Python Treningskurs

Last updated

Kurskode

hadooppython

Varighet

28 timer (vanligvis 4 dag inkludert pauser)

Krav

  • Experience with Python programming
  • Basic familiarity with Hadoop

Oversikt

Hadoop er en populær Big Data behandlingsramme. Python er et høyt nivå programmeringsspråk kjent for sin klare syntax og kodelesbarhet.

I denne instruktørledede, live-trening, vil deltakerne lære å jobbe med Hadoop, MapReduce, Pig, og Spark ved hjelp av Python som de går gjennom flere eksempler og bruker tilfeller.

Ved slutten av denne treningen vil deltakerne være i stand til å:

    Forstå de grunnleggende konseptene bak Hadoop, MapReduce, Pig, og Spark Bruk Python med Hadoop Distributed File System (HDFS), MapReduce, Pig, og Spark Bruk Snakebite for å programmatisk få tilgang til HDFS innenfor Python Bruk mrjob til å skrive MapReduce jobber i Python Skriv Spark-programmer med Python Utvid funksjonaliteten til gris ved hjelp av Python UDFs Forvaltning av MapReduce jobber og Pig scripts ved hjelp av Luigi

Publikum

    Utviklere IT fagfolk

Format av kurset

    Delvis forelesning, delvis diskusjon, øvelser og tung praksis

Machine Translated

Kursplan

Introduction

Understanding Hadoop's Architecture and Key Concepts

Understanding the Hadoop Distributed File System (HDFS)

  • Overview of HDFS and its Architectural Design
  • Interacting with HDFS
  • Performing Basic File Operations on HDFS
  • Overview of HDFS Command Reference
  • Overview of Snakebite
  • Installing Snakebite
  • Using the Snakebite Client Library
  • Using the CLI Client

Learning the MapReduce Programming Model with Python

  • Overview of the MapReduce Programming Model
  • Understanding Data Flow in the MapReduce Framework
    • Map
    • Shuffle and Sort
    • Reduce
  • Using the Hadoop Streaming Utility
    • Understanding How the Hadoop Streaming Utility Works
    • Demo: Implementing the WordCount Application on Python
  • Using the mrjob Library
    • Overview of mrjob
    • Installing mrjob
    • Demo: Implementing the WordCount Algorithm Using mrjob
    • Understanding How a MapReduce Job Written with the mrjob Library Works
    • Executing a MapReduce Application with mrjob
    • Hands-on: Computing Top Salaries Using mrjob

Learning Pig with Python

  • Overview of Pig
  • Demo: Implementing the WordCount Algorithm in Pig
  • Configuring and Running Pig Scripts and Pig Statements
    • Using the Pig Execution Modes
    • Using the Pig Interactive Mode
    • Using the Pic Batch Mode
  • Understanding the Basic Concepts of the Pig Latin Language
    • Using Statements
    • Loading Data
    • Transforming Data
    • Storing Data
  • Extending Pig's Functionality with Python UDFs
    • Registering a Python UDF File
    • Demo: A Simple Python UDF
    • Demo: String Manipulation Using Python UDF
    • Hands-on: Calculating the 10 Most Recent Movies Using Python UDF

Using Spark and PySpark

  • Overview of Spark
  • Demo: Implementing the WordCount Algorithm in PySpark
  • Overview of PySpark
    • Using an Interactive Shell
    • Implementing Self-Contained Applications
  • Working with Resilient Distributed Datasets (RDDs)
    • Creating RDDs from a Python Collection
    • Creating RDDs from Files
    • Implementing RDD Transformations
    • Implementing RDD Actions
  • Hands-on: Implementing a Text Search Program for Movie Titles with PySpark

Managing Workflow with Python

  • Overview of Apache Oozie and Luigi
  • Installing Luigi
  • Understanding Luigi Workflow Concepts
    • Tasks
    • Targets
    • Parameters
  • Demo: Examining a Workflow that Implements the WordCount Algorithm
  • Working with Hadoop Workflows that Control MapReduce and Pig Jobs
    • Using Luigi's Configuration Files
    • Working with MapReduce in Luigi
    • Working with Pig in Luigi

Summary and Conclusion

Testimonials

★★★★★
★★★★★

Related Categories

Relaterte kurs

Kursrabatter

Kursrabatter Nyhetsbrev

We respect the privacy of your email address. We will not pass on or sell your address to others.
You can always change your preferences or unsubscribe completely.

Some of our clients

is growing fast!

We are looking to expand our presence in Norway!

As a Business Development Manager you will:

  • expand business in Norway
  • recruit local talent (sales, agents, trainers, consultants)
  • recruit local trainers and consultants

We offer:

  • Artificial Intelligence and Big Data systems to support your local operation
  • high-tech automation
  • continuously upgraded course catalogue and content
  • good fun in international team

If you are interested in running a high-tech, high-quality training and consulting business.

Apply now!

This site in other countries/regions