Parameterized partial differential equations (PPDEs) arise to describe physical phenomena. These equations often have to be solved either in a multi-query or in a real-time context for different parameters resulting in the need for model order reduction. For transport- or wave-type problems it has been proven that the Kolmogorov N-width decay is poor, such that linear model reduction techniques are bound to fail and nonlinear methods are needed. The recent success in solving various PDEs with neural networks (NNs), particularly with physics-informed NNs (PINNs) suggests that they are a natural candidate for nonlinear model order reduction (MOR) techniques, although an a-posteriori error control is at least not trivial. The goal of our approach is to construct PINNs along with a computable upper bound of the error, which is particularly relevant for model reduction of PPDEs. To this end, we suggest to use a weighted sum of expansion coefficients of the residual in terms of an adaptive wavelet expansion both for the loss function and an error bound. This approach is shown for the linear transport equation using an optimally stable ultra-weak formulation. Numerical examples show a very good quantitative effectivity of the wavelet-based error bound.