The input x(t) and output y(t) of a system are related as y(t)=\int_{-\infty}^t x(\tau) \cos (3 \tau) d \tau . The system is
(a) time-invariant and stable
(b) stable and not time-invariant
(c) time-invariant and not stable
(d) not time-invariant and not stable
Given the relation between input and output as
y(t)=\int_{-\infty}^t x(\tau) \cos (3 \tau) d \tau
To check for time invariance, let x(t) = δ (t). Therefore,
\begin{aligned} y(t) & =\int_{-\infty}^t \delta(t) \cos (3 \tau) d \tau \\ & =u(t) \cos (0)=u(t) \end{aligned}
For delayed input,
\begin{aligned} y\left(t, t_0\right) & =\int_{-\infty}^t \delta\left(t-t_0\right) \cos (3 \tau) d \tau \\ & =u(t) \cos \left(3 t_0\right) \end{aligned}
Delayed output,
\begin{aligned} y\left(t-t_0\right) & =u\left(t-t_0\right) \\ y\left(t, t_0\right) & \neq y\left(t-t_0\right) \end{aligned}
Hence, it is time variant.
To check for stability, consider a bounded input x(t) = cos 3t
\begin{aligned} y(t) & =\int_{-\infty}^t \cos ^2 3 t=\int_{-\infty}^t \frac{1-\cos 6 t}{2} \\ & =\frac{1}{2} \int_{-\infty}^t d t-\frac{1}{2} \int_{-\infty}^t \cos 6 t \cdot d t \end{aligned}
As t \rightarrow \infty, y(t) \rightarrow \infty (unbounded), hence the system is not stable.