Increase sudo timeout

There are two relevant values:

passwd_timeout  Number of minutes before the sudo password prompt times out, or 0 for no timeout.  The timeout may include a fractional component if minute granularity is insufficient, for example 2.5.  The default is 5.

timestamp_timeout Number of minutes that can elapse before sudo will ask for a passwd again.  The timeout may include a fractional component if minute granularity is insufficient, for example 2.5.  The default is 5.  Set this to 0 to always prompt for a password.  If set to a value less than 0 the user's timestamp will never expire.  This can be used to allow users to create or delete their own timestamps via sudo -v and sudo -k respectively.

I find i quit annoying when the prompt times out or you have to type in the password to often.

This are my values:

Defaults timestamp_timeout=30
Defaults passwd_timeout=0

Add them to your sudo conf with visudo.