S3 Read Slowdown with dev branch

1

Hey guys,

I’m seeing close to a 2 second slowdown with the current dev branch when reading from S3 (compared to the 1.3.1 branch). Do you have any idea what could be causing it? (I haven’t looked into it yet, I thought I’d check with you first and see if you had any thoughts).

Thanks!

– Christina

2

Hi Christina,

You may be experiencing reduced parallelism for S3 reads using 1.4 versus 1.3. You can try increasing the sm.num_reader_threads config parameter from its default of 1, and see if that improves performance.

Tyler