Data Ingestion Layer
Data ingestion gathers data and brings it into the data processing systems. The data ingestion layer processes incoming data, prioritizing sources, validating data, and routing it to the best location to be stored and be ready for immediately access. Data extraction can happen in a single, large batch or broken into multiple smaller ones. The data ingestion layer will choose the method based on the situation. It will prioritize a faster loading time and what is best for the program.