The with-block syntax looks like this:

with EXPR as VAR:
    BLOCK

This is so-called syntactic sugar for a much more complex arrangement of try...except and try...finally blocks:

mgr = (EXPR)
exit = type(mgr).__exit__  # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value  # Only if "as VAR" is present
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(mgr, None, None, None)

Which do you prefer?

Few of us would want our code to look this convoluted, but this is how it would need to look without the with statement. Sugar may not be good for your health, but it can be very healthy for your code!