Course Outline
Core Concepts of NiFi and Data Flow
- Contrasting data in motion with data at rest: underlying concepts and associated challenges
- The NiFi architecture: examining cores, the flow controller, provenance tracking, and the bulletin board
- Essential components: processors, connections, controllers, and provenance records
Context within Big Data and Integration
- NiFi's role in Big Data ecosystems, including Hadoop, Kafka, and cloud storage solutions
- Introduction to HDFS, MapReduce, and contemporary alternatives
- Practical applications: stream ingestion, log transport, and event pipelines
Setup, Configuration & Cluster Deployment
- Installing NiFi in both single-node and clustered modes
- Configuring clusters: defining node roles, integrating Zookeeper, and balancing loads
- Managing NiFi deployments through Ansible, Docker, or Helm
Creating and Overseeing Dataflows
- Managing flow operations: routing, filtering, splitting, and merging
- Configuring processors (such as InvokeHTTP, QueryRecord, and PutDatabaseRecord)
- Managing schemas, data enrichment, and transformation processes
- Addressing errors, managing retry relationships, and controlling backpressure
Integration Use Cases
- Establishing connections to databases, messaging platforms, and REST APIs
- Streaming data to analytics tools like Kafka, Elasticsearch, or cloud storage
- Connecting with Splunk, Prometheus, or centralized logging pipelines
Oversight, Recovery & Provenance
- Utilizing the NiFi interface, metrics, and the provenance visualizer
- Designing strategies for autonomous recovery and graceful failure management
- Managing backups, flow version control, and change administration
Performance Optimization & Tuning
- Adjusting JVM settings, heap memory, thread pools, and cluster parameters
- Refining flow architecture to minimize bottlenecks
- Implementing resource isolation, flow prioritization, and throughput regulation
Best Practices & Governance
- Standardizing flow documentation, naming conventions, and modular design
- Security measures: TLS, authentication, access controls, and data encryption
- Enforcing change control, versioning, role-based access, and audit logging
Diagnostics & Incident Management
- Addressing common problems: deadlocks, memory leaks, and processor failures
- Analyzing logs, diagnosing errors, and investigating root causes
- Executing recovery plans and flow rollbacks
Practical Lab: Implementing a Realistic Data Pipeline
- Constructing a complete flow from ingestion through transformation to delivery
- Applying error handling, backpressure management, and scaling techniques
- Conducting performance tests and tuning the pipeline
Recap and Future Pathways
Requirements
- Familiarity with the Linux command line
- Foundational knowledge of networking protocols and data systems
- Previous exposure to data streaming or ETL methodologies
Target Audience
- System Administrators
- Data Engineers
- Developers
- DevOps Specialists
Testimonials (7)
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
James - BHG Financial
Course - Apache NiFi for Administrators
I thought the trainer's pace was good. He left no student behind with his approach. He was very supportive with us NEWBIES that may not have had a System Administrator or Infrastructure role during our career or resurrected those skills from a prior period of our career.
Pamdrea Ivory - BHG Financial
Course - Apache NiFi for Administrators
I like the hands on section. It helped me to better retain information by completing the provided exercises. Also, the trainer's ability to engage with the entire class made me feel comfortable to ask questions on things I was not sure about.
Leila - BHG Financial
Course - Apache NiFi for Administrators
Use-cases, examples for building NiFi dataflows. We worked on troubleshooting common problems and gotchas.
Nelson - BHG Financial
Course - Apache NiFi for Administrators
I loved the structure. We dove into the basics of Nifi, concepts, use cases, etc. on day 1. On day 2, we got to put Day 1 knowledge into practice by building out flows to meet scenario requirements. Day 3 we got to see Nifi Registry and version control, mulit-tenancy, and go over Q&A.
Adam - BHG Financial
Course - Apache NiFi for Administrators
I like how he was able to elaborate about Nifi and how powerful it is. You can basically use it for any infrastructure and use many different computer languages. Also i was glad we were able to fix the Nifi cert renewal issue we were having with the Truststore.
Joachim Martin - BHG Financial
Course - Apache NiFi for Administrators
general knowledge and the possibilities that the training offered in terms on the tool.
