Since you are loading `tcolorbox`, you can use its `listings` library, which seems to automatically take care of the issue that is described below in detail.
```
\documentclass{beamer}
\beamertemplatenavigationsymbolsempty
\setbeamertemplate{frametitle}[default][center]
\usepackage[most,listings,skins]{tcolorbox}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,tikzmark,shapes.geometric}
\usetikzmarklibrary{listings}
\tcbset{
arc=0pt,
outer arc=0pt,
colback=white,
}
\lstdefinestyle{duckstyle}{%
moredelim=[is][\color{red}]{|}{|},
mathescape=true,
escapechar=@,
basicstyle=\ttfamily,
columns=fullflexible
}
\lstset{style=duckstyle}
\newtcblisting{mybox}[2][]{%
colback=blue!5!white,
colframe=black,
fonttitle=\bfseries,
listing options={style=duckstyle,#2},
#1}
%
\newcommand{\Proc}[1]{\textsc{#1}}
\newcommand{\Var}[1]{\ensuremath{\textcolor{varcolor}{#1}}}
\definecolor{varcolor}{RGB}{15,122,183}
\definecolor{titlecolor}{RGB}{29, 110, 174}
%
\newcommand<>\Connect[3][]{%
\only#4{\begin{tikzpicture}[remember picture,overlay,#1]
\draw[->,overlay, dashed] (#2) to [bend right]
([xshift=0.2cm, yshift=.25\baselineskip]pic cs:line-misra-#3-end);
\end{tikzpicture}}}
%
\begin{document}
\begin{frame}[t, fragile]
\frametitle{Pft}
Some text written here introducing the slide to the spectators.
\begin{columns}[t]%
\begin{column}{5.4cm}%
\begin{mybox}[listing only]{name=misra,mathescape,basewidth={.3em}}
Algorithm($\Var{a_1,a_2,\dots,a_m}$)
set $\Var{A}$ = $\emptyset$
For each $\Var{i}$
pft
\end{mybox}
\end{column}
%
\begin{column}{\dimexpr\textwidth-5.5cm} %
\begin{center}
\begin{tcolorbox}[top=0pt, left=5pt,right=5pt, colframe=blue, text width=4cm, text height=0.8cm]
\end{tcolorbox}
\end{center}
\begin{itemize}[<+->]
\setlength\itemsep{5pt plus 1fill}
\item[] $\tikzmarknode{A1}{A} \rightarrow \tilde{f}_A = 1$\Connect{A1}{3}
\item[] $C \rightarrow \tilde{f}_A = 1, \tilde{f}_C = 1$
\item[] $\tikzmarknode{A2}{A} \rightarrow \tilde{f}_A = 2, \tilde{f}_C = 1$
\Connect{A2}{4}
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\end{document}
```
Among other things, this approach also has the advantage that you do not have to tune the width of the `tcolorbox`.
If you do not want to use the `listings` library of `tcolorbox`, things are more complicated. As pointed out in [this post](https://tex.stackexchange.com/questions/164932/problem-with-listings-when-using-line-numbers-name-and-beamer-overlays), there is a problem when using `listings` in beamers: the line numbers get confused. So one can reset them by hand, see also [this post](https://tex.stackexchange.com/a/559303). I also tried to clean up a bit and define a command `\Connect` that is supposed to make life easier, and use `\tikzmarknode` where appropriate.
```
\documentclass{beamer}
\beamertemplatenavigationsymbolsempty
\setbeamertemplate{frametitle}[default][center]
\usepackage[many]{tcolorbox}
\usepackage{listings}
\lstdefinestyle{duckstyle}{%
moredelim=[is][\color{red}]{|}{|},
mathescape=true,
escapechar=@,
basicstyle=\ttfamily,
columns=fullflexible
}
\lstset{style=duckstyle}
\newcommand{\ubar}[1]{\alt<+>{\underaccent{\bar}{#1}}{#1}}
\tcbuselibrary{skins}
\tcbset{
arc=0pt,
outer arc=0pt,
colback=white,
}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,tikzmark,shapes.geometric}
\usetikzmarklibrary{listings}
\newcommand{\Proc}[1]{\textsc{#1}}
\newcommand{\Var}[1]{\ensuremath{\textcolor{varcolor}{#1}}}
\definecolor{varcolor}{RGB}{15,122,183}
\definecolor{titlecolor}{RGB}{29, 110, 174}
\begin{document}
\begin{frame}[t, fragile]{}
Some text written here introducing the slide to the spectators.
\begin{columns}[t]%
\begin{column}{5.4cm}%
\begin{tcolorbox}[top=0pt, left=5pt,right=5pt, colback=blue!5!white,
text width=4.5cm, text height=7cm]
\begin{lstlisting}[firstnumber=1,mathescape, name=misra, basewidth = {.3em}]
Algorithm($\Var{a_1,a_2,\dots,a_m}$)
set $\Var{A}$ = $\emptyset$
For each $\Var{i}$
pft
\end{lstlisting}
\end{tcolorbox}
\end{column}
%
\begin{column}{\dimexpr\textwidth-5.5cm} %
\begin{center}
\begin{tcolorbox}[top=0pt, left=5pt,right=5pt, colframe=blue, text width=4cm, text height=0.8cm]
\end{tcolorbox}
\end{center}
\newcommand<>\Connect[3][]{%
\only#4{\begin{tikzpicture}[remember picture,overlay,#1]
\draw[->,overlay, dashed] (#2) to [bend right]
([xshift=0.2cm, yshift=.25\baselineskip]pic cs:line-misra-#3-end);
\end{tikzpicture}}}
\begin{itemize}[<+->]
\setlength\itemsep{5pt plus 1fill}
\item[] $\tikzmarknode{A1}{A} \rightarrow \tilde{f}_A = 1$\Connect{A1}{3}
\item[] $C \rightarrow \tilde{f}_A = 1, \tilde{f}_C = 1$
\item[] $\tikzmarknode{A2}{A} \rightarrow \tilde{f}_A = 2, \tilde{f}_C = 1$
\Connect{A2}{4}
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\end{document}
```
