The train_datagen object has 3 ways to feed data: flow, flow_from_dataframeand flow_from_directory. The train_generator will be a generator object which can be used in model.fit. The advantage of using ImageDataGenerator is that it will generate batches of data with augmentation. This is available in tf. as ImageDataGenerator class. Keras provides a data generator for image datasets. Since we are reading the images on the go, we are saving memory and even a system with 8GB RAM can be used for a 100GB dataset. So, if we create a data generator, we can read images on the go when they will be used for training. We should be able to use the full potential of available data. Even if it is possible for a small dataset, it won’t be feasible for a large dataset. Have you ever encountered a problem where the dataset you have is too big to be loaded into memory at once that you run out of RAM? For example, while training an image classifier, we won’t be able to load all those images into memory before training. When you write a for loop with range(start, end, step), it does not create a list with all the elements from start to end, but instead, it created a generator that can generate values from start to end and then it will create values on the go. A generator function we use in our day-to-day code would be range function. This way the parent function which invokes the child function does not have to wait until the parent function is done processing but can work on the go. So basically what happens is that instead of doing the hard (computation-intensive or memory-intensive) job as a whole, it breaks it down into batches and work on it as a batch. When a producer function has a hard enough job that it requires maintaining state between values produced, most programming languages offer no pleasant and efficient solution beyond adding a callback function to the producer’s argument list, to be called with each value produced. Photo by Franki Chamaki on Unsplash What are generators?

Imagedatagenerator tensorflow