If each of your examples is large in terms of size and requires parsing and your model is relatively simple and shallow your model is likely to be

I/O bound, so you should look for ways to store data more efficiently and ways to parallelize the reads.

Correct! Your ML training will be I/O bound if the number of inputs is large or heterogeneous (requires parsing) or if the model is so small that the compute requirements are trivial. This also tends to be the case if the input data is on a storage system with low throughput. If you are I/O bound



This is one of the correct answers. dataset = tf.data.TFRecordDataset(…) TF Records are set for fast, efficient, batch reads, without the overhead of having to parse the data in Python.

This is one of the correct answers. dataset = tf.data.TFRecordDataset(files, num_parallel_reads=40) When you’re dealing with a large dataset sharded across Cloud Storage, you can speed up by reading multiple files in parallel to increase the effective throughput. You can use this feature with a single option to the TFRecordDataset constructor called num_parallel_reads.

This is one of the correct answers. shuffle_and_repeat, map_and_batch parallelizes both the execution of the map function and the data transfer of each element into the batch tensors

This is one of the correct answers. dataset.prefetch decouples the time data is produced from the time it is consumed. It prefetches the data into a buffer in parallel with the training step. This means that we have input data for the next training step before the current one is completed.


Leave a Comment