Assignment 2 FAQ

  1. Added March 26 Would it be possible to tell us the vm statistics for the solution code, so that we might have a yardstick with which to measure our own success?

    Sure. I have provided the file refstats.txt, (also linked on the main assignments page), which gives the output of the "vm" menu command for sort and matmult using random and LRU replacement.

    If you cannot run your own experiments, you can discuss the data given here in your lab notebook, referring to the memory reference behavior of these two programs, and the clock and random replacement algorithms.

    For your own experiments, you should try varying other parameters, such as the amount of physical memory and the data set size used by the test programs. (If you cannot run your own experiments, discuss what you would expect to happen as the amount of physical memory, and the data set of these programs increases, and why).

  2. Added March 26 The A2 handout doesn't say anything about design docs like the previous assignments did. Do we need to write any design docs for this assignment?

    You need to (at minimum) provide a proper writeup of your performance experiments. It is a good idea to put this in context by explaining how you implemented random page replacement and your approximation of LRU page replacement.

  3. Added March 21 I can run some user-level tests with my VM system, but on things like forktest, it still fails and I notice the coremap isn't even full yet. What gives?

    The most likely cause is that your swap disk isn't large enough for some of these test programs. If you look around, you will see that a process now gets 512 pages for its user-level stack (USERSTACKSIZE in arch/mips/include/vm.h), swap space for all of them must be reserved when the address space is created (see as_define_stack and trace it down to vm_object_create). However, the default 5 MB disk only has room to store 1280 pages, so any time you try to create more than 2 processes you will have trouble.

    To fix this:

    1. Edit the sys161.conf file in your ~/csc369/root directory and change the line:
             2       disk    rpm=7200        sectors=10240  file=DISK1.img
      
      to:
             2       disk    rpm=7200        sectors=102400  file=DISK1.img
      
    2. Delete the file "DISK1.img" in your ~/csc369/root directory, so that a new file will be created with the new disk size.
    You should now have room for ~12000 pages in your swap file.

  4. Added March 15 The function do_evict takes an integer argument "where", but what is "where"? Is it a virtual address?
    You said in tutorial this week that do_evict has some role in choosing what to evict. So does do_evict just choose a random page to evict and then call lpage_evict to evict the page?

    If you look at what coremap_alloc_one_page (called by coremap_allocuser) does when there are no free pages to allocate, you will see that it calls do_page_replace. That function is the start of the page replacement code. There are two things that do_page_replace should do: (1) select a victim, and (2) evict it. The first will be handled by one of your two implementations of page_replace (either random or LRU-like), the second will be handled by do_evict. So the argument "where" to do_evict is the index of the coremap entry for the physical page to evict. In other words, the value returned by page_replace. The do_evict function is responsible for actually evicting the selected page, and lpage_evict will help with that.

  5. Added March 15 The mmu_unmap function takes an addrspace as an argument and no such argument is passed to lpage_evict. Do we need to unmap the page being evicted?

    If the page being evicted has an entry in the tlb, you will have to invalidate that entry. But, you are right, you do not have the information that mmu_unmap needs to do this. When you choose a physical page to evict using the coremap, you can get a pointer to the lpage that maps it, but you cannot recover the address space that lpage is part of, or the virtual address it corresponds to, so you cannot use mmu_unmap from lpage_evict to do this. I made a mistake yesterday when I told you to use only the mmu_* functions and not use the tlb functions directly.

    You should use tlb_invalidate when you need to remove an entry from the tlb due to eviction. You should still use mmu_map to add an entry to the TLB when you handle a page fault.

  6. Added March 14 How do I add entries to the TLB? Should I be using the TLB_Read and TLB_Write functions as in the old vm_fault function in dumbvm.c?

    The interface to physical structures (RAM or Memory Management Unit and the TLB) is all in coremap.c -- the things you can call from other files are all defined in coremap.h. The functions that begin "mmu_" in coremap.h are the interface to the MMU that you should be using (remember, the TLB is a part of the MMU). These functions call the internal "tlb_" functions in coremap.c. Make sure you understand what each of the mmu functions is for.

    Updating TLB entries in dumbvm.c is pretty bad - a process cannot have more than 64 pages since there is no TLB replacement strategy - once the TLB entries are all filled, dumbvm can't handle the page fault. All of this TLB handling is now provided in coremap.c through the mmu_ functions.

  7. Added March 14 How do you use the LP_SET and LP_CLEAR macros from vmpvt.h? What is the 'am' argument...shouldn't it be 'lp'?
         #define LP_SET(am, bit)  ((lp)->lp_paddr |= (bit))
         #define LP_CLEAR(am, bit) ((lp)->lp_paddr &= ~(paddr_t)(bit))
    

    Yes, it should be 'lp' instead of 'am' in the macro. These macros are very poorly-written (oddly, no one has flagged them before!). The first argument is ignored (not used) and the macro is always applied to the variable named "lp" in the enclosing scope. It works, as long as you really do want to set/clear bits in the lp_paddr field of a variable named lp (as in, for example the lpage_lock function in lpage.c). If you had two lpages, say lp and lp2 and you did LP_SET(lp2, LPF_DIRTY), it would not work properly -- it would set the bit in lp, even though you wanted lp2. Yuck.

    You should change these to:

         #define LP_SET(lp, bit)  ((lp)->lp_paddr |= (bit))
         #define LP_CLEAR(lp, bit) ((lp)->lp_paddr &= ~(paddr_t)(bit))
    
  8. I checked out the a2-branch, built the kernel, booted it, and tried running a user program, but the kernel crashed:
         OS/161 kernel [? for menu]: p sbin/reboot
         sleep: Dropping thread 
         panic: Assertion failed: lock != NULL, at ../../thread/synch.c:139 (lock_acquire)
         ...
    
    What gives?

    You have to modify main.c to bootstrap the swap system. The new vm_bootstrap function now returns the size of main memory, which you need to pass to swap_bootstrap. Also, swap relies on the disk device, and the vfs layer, so both these things need to be bootstrapped first. Use this in your main.c file:

            int memsize = vm_bootstrap();
            scheduler_bootstrap();
            pid_bootstrap(); // DEMKE: initialize pid management before threads
            thread_bootstrap();
            vfs_bootstrap();
            dev_bootstrap();
            swap_bootstrap(memsize);
    

  9. What file does A3 (Strategy) refer to when it says "Look at the code *system161/src/mipseb/mips.c* to see how TLB faults are generated"?

    Sorry... that was a mistake. The assignment page has been updated with more information. Briefly, you would need to unpack the sys161 simulator code, but it isn't critical that you look at how the simulated hardware generates TLB exceptions.

  10. I've looked at the new code and I'm confused. How do I start?

    Start with the tutorial notes from last week (they are posted on the Tutorials page). Remember when you look at the code that anything between "#if DUMBVM" ... "#endif" is NOT included when you build the kernel now.

    There are three things you need to understand (very broadly speaking).

    1. How the virtual address space of a process is managed or described by the array of "virtual memory objects" (vmobj) and the "logical page" (lpage) data structures.
      • the as_objects array in struct addrspace provides the page table for a process. Each region of the virtual address space that is being used by the process (e.g., the program text, the data, the stack, etc.) has a vmobj entry in this array. Each vmobj has an array, vmo_lpages, of lpage structures. These are the page table entries for the process.
    2. How the physical memory space of the machine is managed by the coremap data structure. Each physical page of memory has a coremap entry describing how it is being used. If the page contains part of a user process virtual address space, then the coremap entry points to the lpage in the process that maps the page. Thus, if the physical page is selected for replacement, we can easily find and update the page table entry to indicate that the virtual page it contained has been evicted.
    3. How/when entries are loaded into the TLB by the kernel, and how/when these entries need to be invalidated by the kernel. Remember that the MIPS TLB is software-managed. That means we have the flexibility to use an unusual page table structure, but that OS code is responsible for taking a page table entry and turning it into the correct format for a TLB entry and loading it.

    I think the best way to start is to compile the kernel and run it with the debugger. Set a breakpoint at vm_fault, and then try to run a user program with the "p" command - any user program will do, but for simplicity, try just using the "sbin/reboot" program. Use the debugger to step through what happens as vm_fault calls as_fault, and as_fault calls other functions. (You might also want to stop earlier and look at as_create, as_define_region, etc. to see how the address space is initialized in preparation for loading a program). For the first access to a page, when there are free memory pages still available (without needing to evict anything), most of the work is already done. At the end of as_fault, however, is a call to lpage_fault, which is not implemented. Think about what still has to be done to finish handling the page fault.


Angela Demke Brown
Last modified: Mon Mar 26 12:11:54 EDT 2007