Advanced Operating Systems (Georgia Tech MSCS)

July 20, 2026

Lecture notes from my time in Georgia Tech’s MSCS program. Reference text: Designing Data-Intensive Applications.

Intro to AOS
OS Structures
Virtualization
Parallel Systems
Distributed Systems
Distributed Objects and Middleware
Distributed Subsystems
Failures and Recovery
Internet Computing
RT and Multimedia
Security
Papers

Intro to AOS

Abstractions are interfaces that hide all details within a subsystem e.g. swiping through google earth
- OS provides the software an abstraction for hardware (APIs) so that software never has to worry about directly dealing with it + arbitrates competing requests
For many devices, the hardware underneath has almost the same organization where CPU accesses hardware via bus with Programmed IO or Direct Memory Access
- Can have multiple Buses connected via a bridge to separate higher speed vs lower speed ones
When you click a mouse on an application, device controller raises hardware interrupt on bus, os receives this and passes it to the cpu to handle
OS multiplexes various programs on 1 cpu to give the illusion of multiple programs running in parallel
- It manages system resources, and at runtime of an app, reads from disk and creates memory layout
- Initializes itself at the start and as needed but is out of the way of the program unless some trap function gets called
- Process = program + state

OS Structures

There are a few goals of an OS structure
- Protection across user applications and the OS itself
- Performance minimizing time to execute services
- Flexibility ensuring not one size fits all
- Scalability increasing performance the more hardware resources you provide
- Agility dynamically adapting to applications needs for resources
- Responsiveness minimizing time to respond to external events
Monolithic OS puts every application + OS in its own address space and all OS services/device drivers are contained in 1 big package dropping flexibility for convenience
- What if you want to use a difference page replacement or scheduling algorithm per app?
DOS OS removes user/kernel space boundary making system calls natively fast dropping protection for performance - early days wanted simplicity + single user 1 application (unacceptable)
Microkernel allows flexibility by only making the OS control mechanisms and allowing customizable services for policies in their own address space - services must request resources from the kernel directly
- Can take a performance hit though since need to make various IPC calls to the kernel then to the service and communicate it back vs monolith every service within OS
  - Need to trap/trap back + user/kernel space memory copying
OS structure tradeoffs - can we have all 3?
SPIN Kernel tries to be extensible but also maximize performance
- We want microkernel service owning(OS owns mechanisms), resources without constant border crossings (DOS), flexibility for resource management w/o sacrificing protection and performance
- A few approaches before SPIN were done for extensibility
  - HydraOS had mechanism only OS but capability(token not permission table) based resource access meant that resource managers were coarse grained objects to reduce OS border crossings
    - More coarser the objects are the less opportunity for customization
  - Mach kernel focused on extensibility and portability which lead to overall performance drop
- SPIN colocates kernel and extensions within same hardware address space to avoid border crossings
  - To deal with the lack of protection, relied on strongly typed programming language for compiler enforcement ⇒ logical protection domains established
    - Build kernel in Modulo 3 which has type safety, automatic memory management, and interfaces
    - Access hardware resources via capabilities which are language supported pointers which are type specific
  - Applications can dynamically bind to the same functions leading to flexibility
- To create protection domains must create code with entry points with exported service names, resolve source/target domain bindings if another logical domain wants to use it, and can optionally aggregate domains
- All the above allows kernel and extension to be in the same hardware address space with protection and flexibility/separation
- To deal with external events such as interrupts/exceptions allow events to map to event handlers (1:1, 1:many, many:1) and allow conditionals before executing handler
- SPIN provides core service interfaces/header functions for developers to iterate on
Exokernel decouples authorization from use and acts like a doorman where OS’s bind to the kernel then give it a key and exposes hardware with that key/secure binding
- Establishing the binding (need for hardware) is expensive with a key but using it is lightweight as we separate mechanisms from policies ; for TLB can perform operations on it but only presented to exokernel when hardware needed
  - To implement secure bindings we need to give keys to hardware mechanisms(tlb entry request), need to cache software (shadow tlbs), and can download code into kernel (similar to spin kernel extensions)
- For memory management must page fault to Exokernel ⇒ call OS through registered handler w/ key ⇒ OS handles fault and presents mapping to Exokernel with key, and Exokernel installs mapping on TLB
  - For every process store a software TLB to preload hardware TLB on process context switch since hardware TLB is flushed every context switch
- Exokernel allows downloading code into kernel w/ secure binding but how can this be secure? ⇒ Only allow trusted set of users with this privilege
- When Exokernel wants to revoke resources, it will issue a revoke call with a repossession vector of revoke details to reclaim all resources needed - can also be notified ahead of time that resources will be revoked so it can let exokernel save them beforehand
- Exokernel flow for multiple OS’s + Process Environment data structure per OS for handling any events
- When measuring performance compare against bad performance microkernel with best extensibility with good performance monolithic kernel with worst extensibility
  - Find that SPIN + Exokernel are much better than microkernel in performance and as good of performance as monolithic kernel
L3 Microkernel can we achieve good performance with a microkernel based structure compared to Mach which compromised performance for portability?
- 2 main factors of performance loss in a traditional microkernel
  - Border crossings to kernel then to multiple services (each in own address space) then back to kernel then back to user space takes time
  - Interservice to/back kernel communication ~= Protected Procedure Call is 100x slower than normal procedure calls due to cache locality and lack of entires in TLB
- L3 tries to debunk microkernel myth(strikes) saying its how you implement the microkernel not that if you follow microkernel pattern its over
  - All strikes will be debunked
    - Kernel user switches for L3 takes 123 processor cycles (includes TLB and cache misses) vs original bad Mach 900 cycles
    - Normally on context switch we have to flush entire TLB - however if we are context switching between different services via Protected Procedure Calls (no address space change), we can use address space tagged TLBs to detect if we switched address spaces or not (preserve TLB entries)
    - If possible take advantage of hardware and use segment registers for protection domain where every segment explicitly enforces bounds and no need to check TLB first
      - If services take up entire address space then forced to flush TLB and you see the implicit costs (800 cycles of cache effects) dominate over explicit costs
      - Address space switching should be done on small protection domains to minimize TLB flushes as implicit costs dont dominate
    - For cache locality loss if protection domains are small, store them in the same hardware address space to prevent cache flushing - for big ones they will always be flushed regardless of kernel type
- The reason for Mach’s expensive kernel/userspace/service traps is due to its focus on portability which sacrifices performance
  - Since needs to support multiple architectures has a lot of code bloat which leads to larger memory footprints which leads to less cache locality which leads to more cache misses which leads to higher latency
- Thesis of L3 is to have minimal abstractions, be processor specific for every microkernel leading to non portable (maximized performance) implementations, and the combination of the 2 leads to efficient processor independent abstractions at higher layers

Virtualization

Virtualization lets any application/OS run on any/shared hardware and gives the illusion of having their own platform ⇒ cost and maintenance much cheaper
- Resource usage occurs in bursts so can combine (and bill) multiple smaller clients in 1 big shared server
Hypervisors intercept all OS requests and deal with the hardware management
- Either has hosted on top of guest os which is more flexible or bare metal on top of hardware maximizing performance
Full virtualization involves the untouched OS being run as user level programs and whenever privileged operation occurs, it traps into the hypervisor and hypervisor deals with operation
- Some system calls may fail silently since not supported in virtualization so hypervisor must take that code and translate into an executable version (binary translation) and was how VMWare was formed
Para virtualization involves directly modifying source code(~2%) to optimize virtualization operations and avoid unsupported instructions
Memory virtualization give OS illusion that it owns the memory in virtualized environment
- Recall in paging we have page table entries mapping to different parts of the address space
- Hypervisor does not manage page tables only performs the necessary hardware related privileged operations relating to them
  - Now with virtualization we have Virtual to Physical to Machine Memory abstraction where machine memory is the only contiguous memory available
  - Hardware page table in this case is the shadow page table
    - If we need more physical memory but no more machine memory then we can either forcibly steal memory from another OS or ask politely to another OS for memory
- For full virtualization we go from VPN → PPN → MPN every access but can optimize this by mapping the VPN alongside PPN directly in shadow page table to MPN inside TLB
- Paravirtualized we can move the page tables into the OS to handle VPN to MPN mappings and update page tables via hyper-calls
- To keep up with increased memory demands in a full machine memory system, hypervisor can install balloon drivers beforehand per OS and these drivers can talk to other drivers in order to inflate(get more memory) or deflate(free more memory)
- If using same OS, same applications, same operations etc… can share machine memory across OS’s via copy on write pattern
- For VM Oblivious page sharing, hash the content of the page table and record PPN’s and VPNs for that given hash
- Can use different memory allocation policies per virtualized environment
  - Can share AND/OR assign memory to groups of environments
  - Implement a tax rate per idle pages and reclaim them once tax rate limit hit
CPU virtualization give OS illusion that it owns the cpu in virtualized environment
- Now instead of managing the processes (which the OS manages independently) the Hypervisor must manage scheduling OS’s like OS’s manage scheduling processes
- When running on CPU in virtualized environments, every memory address is translated
  - We may get events like sys-calls/page faults etc that need to be delivered to the OS as interrupts
  - However to deal with certain privileged operations, they may fail silently if user is run in user mode so hypervisor needs to go directly in binary and translate it
Device virtualization give OS illusion that it owns the hardware in virtualized environment
- Full virtualization do a normal trap and emulate
  - Trap to hypervisor and send interrupts (events) to OS w/ no control when to send
- Para virtualization OS directly sees hardware so more chances to optimize via giving shared buffers for device to use and directly receiving events as needed
  - Hypercall to hypervisor and send interrupts (events) to OS w/ control when to send
- Data transfer for full virtualization is implicit but for para virtualization we can optimize hardware IO calls a bit via producer/consumer requests and responses by OS guests and consumer Xen CPUs
  - Request producer by OS moves clockwise and request consumer Xen takes data segmented by file descriptor/id and processes
  - Response consumer by OS also moves clockwise consuming any IO events sent to OS produced by response consumer
- For network transfers for transmitting OS’s provide buffer locations to Xmit ring separated by file descriptors and receive these buffers in a recv ring where no copying occurs as receiving end preallocates buffers to put data in or swaps recv page with another page
- Disk IO virtualization similar to network virtualization
- Goal for virtualized environments is focus on protection and flexibility along with tracking usage for billing purposes

Parallel Systems

Shared memory machines have many different versions
- Dance Hall Architecture where cpu + cache connects to memory via interconnect
- Symmetric multiprocessor/Shared Memory Machine Model where cpu + cache connects directly to shared memory (normal multicore cpus)
- Distributed shared memory in order to talk to another cpus shared memory need to travel over the interconnect
- Assume that all cpus have value x in their cache, if 1 updates x how do we update x in the SMP model ⇒ cache coherence problem
  - Memory consistency to the user and cache consistency to the OS are intertwined
  - Sequential consistency preserve program order (if write to x happens before read of x then read of x reads the write of x + arbitrary interleaving instructions)
  - Cache consistency if update x then write invalidate invalidates x on every other processor and write update will update x on every other processor with new value
- If we add more processors we want linear scaleup as we exploit paralellism but need to keep in mind that the more shared memory we use, the more overhead is incurred
Many ways to improve upon synchronization primitives such as locks and barriers
- Need atomic operations to implement a lock specifically following the fetch and phi pattern
- 3 factors into determining synchronization performance: latency, waiting time, contention
- Naive spinlock has too much contention, does not exploit cache (goes to main memory) and disrupts useful work
- Caching spinlock spins on cached version of lock in SMP system but when unlocked all n processors do test and set instruction causing cache invalidation to happen n^2 time which is disruptive
- Can implement a delay after lock release but implement exponential backoff in order to make processors not all come for the lock at the same time
- Ticket lock assigns numbers to processors which increases fairness but increases contention
- For all these locks, whenever a lock is released all threads try and see if they get the lock next - why not let the current holder of the lock signal the next holder for a queueing lock
- Array based queuing lock initialized array of size Z and everyone waits on only its current spot which reduces contention however space is O(n) so could be large in some cases
- Linked list based queuing locks saves space proportional to number of requests of lock
Barrier synchronization involves threads waiting at a specific point in a program before resuming execution
- Centralized barrier works but all processors go to next barrier when last processor sets count to N
- Sensing barrier makes processors spin on sense flag where last one reverses the flag in order to avoid 2 spin blocks ⇒ all processors spin on 1 shared variable increasing contention
- Tree barrier aims to reduce contention as you spread the amount of locksense variables to spin on logarithmically
  - Processor will go to its corresponding count, if it is not 0 after decrement spin else recursively traverse upwards and repeat
  - If root set to 0 then all processors at root recursively go down the tree, flip the locksense variable, and repeat down the tree
- MCS Tree barrier is predetermined children where parents spin on vs dynamic tree barrier where any processor can move up/move down and signal with 4-ary children
  - Children spin on predetermined wakeup tree which is binary in this case
- Tournament barrier preassigns winners for losers to spin on every level and once we reach the root, winners gradually notify losers down the tree
  - Works well on NUMA processors as they can spin on a value close to the processor and doesn’t need SMP system
  - Works in a distributed memory context as can assign winners/losers + notify just by using messages
- Dissemination barrier is message based gossip protocol with no hiearchy where nodes send each other details about nodes that are spinning/free in log n rounds
For RPC’s on the same machine we have to decide between trading off performance vs safety
- Procedure call stuff all happens at compile time vs RPC at runtime where it must constantly copy data to and from the kernel to execute the RPC function w/ 4 copies each way
- In order to make and RPC from client to server, must get granted access from the kernel which contacts the name server
  - Kernel creates data structure and shared memory space to transfer arguments which is expensive but binding is 1 time cost
  - Use an A-Stack and pass in data vs shared memory in order to go from 4 to 2 copies
    - Keep in mind that switching protection domains to set everything up here will lead to loss of locality
- For SMP’s can preload server domains to keep caches warm and avoid locality losses
Scheduling involves picking the best processor possible but many heuristics to consider - what about picking the one with the most amount of memory in cache
- Need to watch out for interleaving threads that may pollute the cache (cache affinity scheduling)
- Different scheduling policies
- Minimum Intervening policy assign cache affinity index to threads per processor where further away in time threads have bigger affinity indexes and assign lowest index processor for a specific thread
- Minimum Intervening queue policy is same as above but add in number of items in queue since could take a while for selected thread to run
- Use local cache affinity queues in order to reduce global queue overhead at scale
- Need to consider various aspects of performance in scheduling
- Cache aware scheduling make sure all threads cache requirements (even ones not scheduled on the cores) are less than the size of L2 cache
Tornado is a shared memory multiprocessor OS
- Principles include efficient cache utilization and limiting contention on shared data (instead replicate/partition these shared data structures)
- Summary of what goes on in a page fault
- Easy to parallelize multiprocess workload pagefaults as page tables are distinct vs multithreaded workload
- Tornado uses illusion of single object which has multiple representations under the hood
- For memory management, we can split the page table into individual file cache managers which cover certain parts of the virtual address space
- Maintain location table of object references to representations per cpu, if no representation found then miss handling table either connects reference to existing representation or new one, and if no reference available in miss handler go to global miss handler and install
- Instead of locking everything (hiearchal locking), use refcounts for existence and split up data to maximize concurrency and minimize locking
- Can break up heap for scalable dynamic memory allocation
- IPC is done with Protected Procedure calls - no context switch locally and full context switch remotely
- In the corey system, the app gives all the address ranges its operating in, it tells the OS which structures are shared or not, and has dedicated cores for kernel activity
- Can do virtualization on multiple operating systems with cellular disco layer
  - When interrupts happen, cellular disco intercepts it and passes it to host OS instead of directly sending to host OS

Distributed Systems

A distributed system is a system where the the message transmission time is not negligible to the time between events in a single process + no physical memory shared between nodes and nodes are connected via some interconnect
- Assume within a process, events are ordered and between processes send happens before a receive
  - a → b implies a before b in single processor or a → b send/recv in multiple processors
  - a → b and b → c implies a → c
  - a || b concurrent events cannot say anything about a and b’s ordering
Lamports clocks are a way to reason about clocks in a distributed system
- Logical clocks track on their own process and received message times should be the max of the sent time or the current processor time
- Lamports logical clocks track partial orders but what if we want total orders - ie you want nodes to access some global state based on timestamp but all timestamps are locally generated
  - Total order can be done with a tiebreaker function
- How do we implement a distributed lock algorithm assuming messages arrive in order and there is no message loss? ⇒ send messages to every other processor to put in “happens before” queue and send back ACK’s and claim lock if on top + received ACKS OR top of queue and all lock requests underneath lower priority
- Logical clocks are not good enough as individual processors have clock drift and can synchronize to different clocks
  - Clock A is less than Clock B on another processor if the drift between the 2 is less than some small k
  - Avoiding anomalies should make sure the mutual clock drift is less than IPC time (M)
- Should have bounds on mutual clock drift and individual clock drift
Latency and throughput are related somewhat but need to be dealt with individually
- There are many components of rpc latency: marshaling, data copying, control transfer, and protocol processing
- Can reduce marshaling and data copying by marshaling directly into kernel buffer(unsafe) or shared descriptors between the client stub and kernel
- Can reduce context switch by overlapping them while sending data over the wire + spinning over context switching on the client side
- Many ways to reduce latency if connected by a LAN
How can we route a packet efficiently over the network and make it customizable
- Router tables execute code and figure out the next hop to route packet too ⇒ active networks
  - Helps to implement multicast, congestion notifications, private ip, any-casting
- OS can intercept the code and os can check/execute code but this is a lot of overhead since need to modify the protocol stack and not all routers have the ability to do this
- ANTS allows the application to add a ants header so the OS is undisturbed - if the node the packet lands on is stateless then use IP header else ants header
  - Also make active nodes on the border 1 away from destination so the core IP network is undisturbed
- When an active node receives a packet, use the md5 hash and get the capsule from the previous node - if not there then drop the capsule similar to IP routing packets
- Active networks can only be done in the edge of the networks since vendors don’t want to open them up within their routers, software routing will always have lower performance than hardware routing, and a bit uncomfortable for arbitrary code executing in router
How can we build systems with a component based architecture
- Spec out with IO Autometer, Produce code with OCAML, and Optimize with NuPrl

Distributed Objects and Middleware

Spring OS is an application of a distributed OS for commercial use
- When innovating OS should we create it brand new or iterate upon an already known one? ⇒ Already known OS is better to not reinvent parts of the wheel (Intel/Unix inside)
- Object based design modified state indirectly via called methods vs Procedural design which can modify the state however it wants
- Spring is built on strong interfaces where the microkernel has the bare minimum and all other interfaces are defined with IDL and can be implemented in any language
- Nucleus the microkernel manages the threads and rpc and requires door handles to access doors to execute protected procedure calls on target domains
- In order to access doors on another machine, request travels over proxy to execute
- In order to enforce permissions can use front objects to state number of times request can be accessed w/ permissions or full access
- Memory is stored in regions where memory objects contain multiple regions in Spring
- For DRAM establish physical to virtual address space by intermediary pagers which link to memory objects
- Subcontracts allow clients to dynamically bind to servers where they handle invoke calls and internally handle permissions
Java RMI allows objects in one JVM to invoke methods on another object in another JVM
- Can reuse local implementation and extend to remote interface
- Can reuse remote interface which puts the load on Java’s runtime over the implementer
- Code implementation
- Actual RMI logic(serialization and deserialization) is handled by Remote Reference Layer and decides which transport channel to use
Enterprises consist of multiple machines within an enterprise and are also connected with multiple other enterprises as they grow which makes these systems exponentially complex
- Opportunity to parallelize queries and their execution for N tier applications
  - Can place beans within containers to handle logic

Distributed Subsystems

Global Memory Systems: How can we use peer memory for paging across the LAN
- Normally virtual address space bigger than physical address space so need to access disk for new pages
- However what if there are other idling processes on a LAN which allow faster remote memory access - can we access cluster memory instead
- GSM Definitions: cache is physical memory not processor cache ; local memory is the working set and can be shared and global memory is spare memory that can be used by other nodes ; for page replacement must coordinate across all nodes for oldest page LRU
- Page faults need to be handled in various cases
  - Case 1: node page faults, sees page on other cluster and needs to swap in any page in global set for this cluster page
  - Case 2: node page faults, sees page on other cluster and due to memory pressure needs to swap oldest page in local set for cluster page
  - Case 3: node page faults and locates on disk which means global part shrinks and needs to page something out - on another node if oldest page is global then implied clean else needs to write out to disk which allows global portion more memory
  - Case 4: Case 3 but shared memory pages (don’t evict local pages here)
- Idle nodes eventually fill their entire working set with global memory ; if it starts up again then local memory will fill up and global will shrink
- For LRU eviction, for every epoch we define 2 parameters T the max duration of page and M the max page replacements per node
  - Nodes send to initiator the pages and their ages and initiator sends back the min age parameter and weight ie % of pages to evict
  - Per node if evicted age of page X > minAge then discard else send to a peer - can determine idle page with weight param as higher weights mean more pages that are being evicted
  - Next epoch initiator is node with the highest weight
- Unix implementation and integration - GMS only deals with reads and does not deal with writes as that could compromise the system
- Data structures: convert virtual address to uid → hit up Page Ownership Directory for specific node → hit up Global Cache Directory for finding the page frame directory → hit up page frame directory for the page frame number for distributing load
- If page miss due to data structures either POD is changing due to addition/deletion or PFD hasn’t notified GCD about node change common in distributed systems
- If page eviction, POD sends update node request to GCD and POD puts page in PFD node
Distributed Shared Memory: How can we make the cluster appear like a shared memory machine
- Can use cluster as a parallel machine with sequential program by user written directives known at compile time but limited potential for exploiting available parallelism
- Can use cluster as a parallel machine with message passing framework like MPI
- Can use cluster as a parallel machine with DSM memory framework where processors use shared memory exactly the same as regular memory w/o any message passing/directive overhead
- Hardware/Software/Structured DSM systems have been built since the 80’s
- 2 types of memory accesses: reads/writes to shared data and reads/writes to synchronization variables where normally you don’t distinguish
- Memory Consistency deals with the model presented to programmer which software is written and cache coherence deals with the hardware implementing the model w/ private caches
- If 2 processes lock and access the same variables do we really need to maintain cache coherency for both inside critical section → no as only 1 accesses it → release consistency mandates coherence actions after releasing lock and before new thread acquires lock
  - Distinguishes between data and synchronization accesses and also overlaps computation with communication
- Lazy release consistency involves performing coherence operations on new thread acquisition compared to after previous thread release
  - Eager RC push model so broadcasts variable coherency changes to all processors vs Lazy RC pull model so only reaches out to previous thread coherency changes
- Software DSM provides global virtual memory abstraction and when a page fault for shared memory happens, DSM contacts owner of the page and also makes sure that (separate) processor’s page ok to share and then pages this in to the requesting processor
  - However when need to write to page, DSM invalidates other processors shared pages beforehand and then allows write which is expensive and could lead to false sharing
- LRC with multiple writers coherence allows processors to invalidate pages that are attached to a specific lock - if multiple processors modify these pages then these changes keep propagating to other processors that acquire the lock
  - If another processor modifies variables corresponding to a different lock then we only invalidate pages within that specific lock hence multi writer coherence
- Implementation when creating new page keep twin for difference calculation + garbage collect old diffs in order to reduce latency in sending diffs to new writer process
- If using non page based DSM then need to track individual reads and rights or hit up an API and whenever accessing shared variable must trap into kernel to perform cache coherence
Distributed File System: How can we use cluster memory for cooperative caching of files
- Network File System
- Distributed File System distributed files to maximize throughput on files
- Distributed file systems built upon striping, log structure, and RAID
  - Striping RAID technique which increases IO bandwidth by splitting up data into multiple disks but prone to failure if one fails (caught with checksum on different disk), expensive for multiple disks, and 1 small write could write to all disks
  - Log structured file system buffers changes to log segment data structure which gets flushed periodically or when full to disk
  - Software RAID combined LFS + RAID by striping file across multiple LAN connected disks
- XFS is a distributed file system that uses log based striping, cooperative caching, dynamic data/metadata management, subsetting storage servers, and distributed log cleaning
- XFS dynamically distributed metadata and does cooperative client file caching
- XFS uses a log buffer for log based striping and only write to a determined stripe group amount of disks
- XFS implements cooperative caching by storing metadata and details in memory and invalidating files on writes and then allowing the write to happen
- When we write to files, we overwrite old places in the log segments and overtime we will coalesce the latest updates and garbage collect the old log records
- Unix file system given {filename, offset} → inode → data blocks on disk
- XFS Data Structures
- Client reading from file if not accessed by another client goes to unix cache, if accessed by another client must go to mmap directory and access peer unix cache for file, else long path
- Client writing to file writes to all disks in stripe group and notifies manager

Failures and Recovery

Persistence is needed in order to save data you updated permanently ~= Lightweight Recoverable Virtual Memory
- Imagine making virtual memory persistent so every update of virtual memory needs to be written to disk which causes lots of random IO calls → instead can we write to a log segment and buffer these changes?
- We don’t need to make all of virtual memory persistent → the designer can choose which virtual address regions are persistent and map to same or multiple data segments to back persistence
- Recoverable Virtual Memory allows you to initialize by region, do transaction management with address range specification, and provides the options to flush/truncate the log file manually
- When starting transaction, create an undo record just in case to reverse any aborts/error modifications, and if transaction commits then override with a redo log
  - Can enable no-restore mode at beginning of transaction to prevent creating an undo record (assumes that transaction will never abort/error out)
  - Can enable no-flush mode in end transaction to prevent blocking due to log file not being written to disk yet (helpful if minimal power failures)
- If crash happens, read from end of log first and apply changes
- Log redo records may be piling up in the file, so in parallel can read log and apply updates from the start and define epochs within log records to indicate when it can be applied to disk
RioVista asks can we eliminate synchronous disk IO compared to LRVM?
- System can crash with power or software failure ⇒ What happens if we eliminate power failure with a power supply + persistent memory and only have to deal with software failure
- Instead of writing directly to disk we can write to a power backed file cache
- When committing transactions, we can do so asynchronously as already in file cache so non blocking and if aborted/crashed then we can use our mmap’ed undo log to recover
Quicksilver asks us if recovery is so important, why don’t we prioritize it?
- Quicksilver structured like traditional microkernel and first to implement transactions for recovery management
- Quicksilver offers synchronous/async client calls alongside multiple server waiting with service queues
- Transaction managers connect with each other to track state on client/server calls

Internet Computing

OSI Model
- Can load balance at the network level where you cannot distinguish between requests of the transport level where you can
- DQ principle states that the server can accept a limited number of requests (yield) and can gather a limited number of data (harvest)
  - In order to increase the number of requests, you should expect lower data and vice versa for yield and harvest relationship
  - Replication lowers yield and Partitioning lowers harvest when failures happen ; the rest go unchanged ⇒ allows us to gracefully degrade services by keeping 1 parameter fixed and reducing the other
- Multiple ways to bring down servers in order to upgrade them
MapReduce deals with computing results on big data clusters
CDN utilize a distributed hash table in order to give a fast, reliable way to access locations of content
- We hash key(content) and value(ip addresses) pairs and place these on nodes where keys are approximately the node’s value
- Use an overlay network in order to route requests with next hop table
- Can assign and find locations via greedy approach where place at key N or find key closest to N in order to find its location
  - However ends up into the metadata server overload problem: all keys close to N will end up on the same metadata server and all requests for a particular resource close to N will all route to the same metadata server
    - Can solve with web proxy which caches content but not good enough for live content
    - Can solve with a CDN provider which mirrors content globally and requests get rerouted to local server but expensive ⇒ Coral System democratizes content
Coral DHT satisfies requests from nodes different from the same/similar key with key based routing: xor the distance between src and dest to avoid congestion and spread metadata around

RT and Multimedia

We need real time guarantees sometimes in an OS in order to bound latency
- A timer event resulting in an interrupt later can be delayed many ways increasing latency
  - Has timer inaccuracy latency, time preemption latency where kernel does something, and scheduler latency where another processor runs
- Multiple timers: periodic timers normal but interrupts can be delayed, one shot timers accurate but have overhead, soft timers the kernel polls the application reducing overhead but increasing polling overhead/latency ⇒ firm timer is what TS-Linux uses combining benefits of above
  - Overshoot parameter in firm timer allows for timers to be configured during any arbitrary kernel trap
  - Store timers in sorted linked list, use APIC hardware to reprogram timers in a few cycles, and if kernel knows one shot timer will need to be reprogrammed in future, reprogram it in a previous periodic dispatch
- To reduce kernel preemption latency allow preemption only when kernel not modifying kernel data structures to prevent race conditions ⇒ explicit lock for data structure modification
- To reduce scheduling latency use proportional period scheduling and avoid priority inversion by letting server service times be max priority
Persistent temporal streams
- Parallel programs use threads and distributed programs use sockets which sometimes too low level
- Lots of multimedia apps are sensor based which leads to lots of distributed data streams that needs to be
- Can group PTS streams together and get all their stream data together

Security

Computer Security is very important in an OS
- 4 levels of protection in security: unprotected (only tracks mistakes), all or nothing (communicate only with IO), controlled sharing (access lists), user programmed sharing controls, user defined strings
- Build the system to detect violations over prevent as much easier and don’t be too conservative with system security rules relax them a bit
Andrew File System is a distributed file system that has security protocols over the network
- How to authenticate users messages, server messages, prevent spoofers from making false requests (replay attacks), and isolate users from one another
  - Only use username and password for logging in and use ephemeral id and keys for communication later on
  - 3 types of client server interaction in Andrew: logging in, establishing RPC session, file system access during session
- Provide username and password for logging in, get back clear and secret tokens where secret token is used as id for sessions and handshake client for decryption
- Establish RPC session with client sending random id, server decrypting with Handshake Key Client, sends back random id + 1 to establish server genuine, client decrypts with Handshake Key and sends back server id + 1 for client genuine after server decryption
  - Server sends back session key to authenticate along with starting sequence num for RPCS
- Workstation links are insecure and file system links secure so RPCs to and from the filesystem are encrypted
- We encrypt data using public keys and can only decrypt it with a private key held on the file system servers where both are one way operations
- Summary