Yep - the two-node cluster can continue operating while the File Share Witness is offline for 3 - 4 hours, provided the two cluster nodes remain healthy and can communicate with each other. The FSW is not required for normal cluster operation when both nodes are online. Its purpose is to provide the additional quorum vote needed to avoid a split-brain situation if communication between the two nodes is lost.
Before taking the FSW offline, verify that both cluster nodes are online, communicating normally, and that the cluster currently has quorum. Also verify that the FSW is currently functioning and that there are no existing cluster warnings or quorum issues. I would not deliberately place the cluster into a special "maintenance" or paused state simply because the FSW is being upgraded.
During the maintenance, the cluster will effectively operate with the two node votes while the FSW vote is unavailable. With both nodes online, the two-node cluster retains quorum and should continue operating normally. The important restriction is that you should not reboot, shut down, or otherwise make either cluster node unavailable during the FSW maintenance window. If one node fails while the FSW is offline, the remaining node would no longer have sufficient votes to maintain quorum, and the cluster could stop its clustered roles.
So confirm healthy quorum and node-to-node communication, take the FSW workstation offline, perform the BIOS, driver, and Windows 11 upgrades, and bring the workstation back online afterward. Once it is back online and connected to the domain/network, verify that the File Share Witness is again online and that the cluster reports normal quorum. There is no inherent timeout after which an offline FSW causes the cluster to shut down; the critical issue is losing another cluster vote while the witness is unavailable.
As an aside, you should not cluster AD domain controllers - two DCs provide sufficient high availability without Failover Clustering
If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.
hth
Marcin