Formulating and running a Decision Optimization model: house construction scheduling

The problem

You need to plan and schedule activities and subcontractors for a house construction project. Your schedule must start on a particular date. All the activities (masonry, carpentry, plumbing and so on) must be scheduled and there is a specified order of activities that must be respected (for example windows cannot be put in until the roof is completed). Each subcontractor can perform some of the necessary activities and with differing level of skills. Your schedule must determine the best (earliest) end time for the construction project ensuring that all activities have been scheduled and decide which subcontractor to assign to each activity. In addition, you would like to know how to optimize the skill level of your subcontractors on this project.

Your data

You have data for this project as shown in the following spreadsheet. For each activity you have the duration that is needed to complete it, the activities that must precede it and the possible subcontractors who are available and qualified to perform that activity.

For illustration purposes, there are just 10 activities and 3 subcontractors shown. With Decision Optimization it is easy to change your data and solve the same problem with larger data sets.

For each activity you also have data concerning the level of expertise that each subcontractor has for that activity. The higher the number, the more expertise the subcontractor has. If a subcontractor has a zero skill level, he must not be assigned to the task. The following table shows part of this spreadsheet.

You also have a table containing the names of the Subcontractors (Joe, Jack and so on) available for this project.

Obtain data files for this example

The data files used in this example are available in the DO-samples. Normally, you would have your files already stored in a project as a data asset or locally on your machine. For illustration purposes however, so that you can build the model yourself, in this example you will first download the data files onto your machine and then import them into the project that you have just created. The completed model formulation with imported data is also provided as a sample, see HouseConstructionScheduling in DO-samples.

1. Download and extract all the DO-samples on to your machine.
2. Open your project.
3. To open the data pane, click Add data +.
4. Select Drop data files here or browse for files to upload.
5. Browse your downloaded DO-samples to locate the house_activity.csv, house_expertise.csv, and house_subcontractor.csv in the datasets folder .
6. Click Open. The files are uploaded as data assets in your project.

Create a Scenario

Requirements

To edit and run Decision Optimization models, you must have the following prerequisites:

Admin or Editor roles

You must have Admin or Editor roles in the project. Viewers of shared projects can only see experiments, but cannot modify or run them

Machine Learning service

You must have a Machine Learning service that is associated with your project. You can add one when you create a Decision Optimization experiment.

Deployment space

You must have a deployment space that is associated with your Decision Optimization experiment. You can choose a deployment space when you create a Decision Optimization experiment.

To create a Scenario:

1. Open your project or create an empty project.
2. Select the Assets tab.
3. Select New asset > Solve optimization problems in the Work with models section.
4. In the New Decision Optimization experiment window that opens, enter a name.
5. If you haven't already associated a Machine Learning service with your project, you must first select Add a Machine Learning service to select or create one before you choose a deployment space for your experiment.
6. Click New deployment space, enter a name, and click Create (or select an existing space from the drop-down menu).
7. Click Create. A Scenario 1 is created along with the model, and you work in Scenario 1.

Your Scenario specifies the combination of data and the optimization model formulation that you want to solve. You can create different scenarios with different variants of data and model formulations.

Prepare data

The experiment UI opens displaying the Prepare data view. The data files that you have in your project are displayed in the data pane. (If necessary, click to open the data pane.) Select the three house sample files and click Import.

The data files you imported are now displayed as tables in the Prepare data view. The following image shows the data files house_activity.csv, house_expertise.csv, and house_subcontractor.csv imported in a Scenario.

You can view all the data by scrolling in a table. You can also view all the data by clicking the Open the table in full mode icon of a particular data table. You can edit data values directly in the table as well as in full mode.

Choose the Modeling Assistant

1. Click Build model in the sidebar and a pop-up window appears asking you how you want to formulate your model (whether you want to use the assisted mode with the Modeling Assistant or create or import a model in Python, OPL, LP (CPLEX) or CPO code.
2. Select Modeling Assistant.

Define your decision domain

In the Model view: select the decision domain for your problem. In this case select Scheduling. The decision domains currently defined are Scheduling, Resource Assignment, Selection and Allocation and Supply and Demand domains.

1. After selecting your domain, a pop-up window appears for you to map your data to the scheduling concepts Tasks and Resources. Tasks are whatever you want to plan and schedule over time. You must define at least one task to be scheduled. In this example, your tasks are construction activities such as masonry. Resources can be human, machine, equipment or anything you want to use for the tasks. In this case your resources are your subcontractors.
2. Under TASKS, click Choose a task and choose house_activity from the drop-down list. Then under RESOURCES click Choose a resource and choose house_subcontractor. The names of possible tasks and resources for you to choose from are taken from your imported data. For this example, you only need to map activities and subcontractors, but you could add other tasks and resource mappings if your model required it. You can remove any mapping by hovering over it and selecting the delete icon.
3. Click Continue.

How tasks will use resources

In the window, for each task to be scheduled, you have three options :

For this example:

1. Choose the default setting with all options (Use the resources... While assigning...) selected and click Continue.

   The problem that you want to solve is now formulated in a concise statement.

2. Click Finish.

   You return to the Model view. You can edit your problem definition again at any time, by clicking the Edit intent (pencil) icon and redefining your mappings and scheduling options.

Your model formulation

Now that you have specified the problem that you want to solve, the Modeling Assistant provides you with a partially completed formulation in this model view. The Objectives and Constraints pane contains the model that you will run. The Add to model pane, contains more suggestions that you can include in your model formulation. If you have re-sized your window, it is possible that the Add to model pane appears underneath the Objectives and Constraints pane.

The model consists of an objective to be attained (maximized or minimized) and some constraints that must be satisfied. For scheduling problems like this, your objective is to work out the best schedule. The best, in this case, is one in which the time to complete all the activities is minimized. (You want to complete the house construction as quickly as possible as this will reduce costs.) This objective as well as some standard scheduling constraints have been automatically added to your model. You can also use the Objectives search field to search the objectives and constraints.

It is possible that your constraints are displayed in a different order. There is also a constraint that is automatically added to all scheduling problems with assignment. This enables you to accept or refuse to assign subcontractors who have unavailable periods during the activities that are scheduled. In this example, unavailable periods are not considered so leave this constraint as it appears by default.

Some constraints have more details that can be displayed or hidden by clicking the arrows on each line. A bar next to the constraint indicates that there is a value or definition that you must add. You can add items by clicking the term shown underlined and typing in or selecting from a drop-down list, and you must complete the model before running it, but before doing this, first save a copy by duplicating the scenario as explained later in this section.

In the model view of your scenario, if you click the Replace icon next to Modeling Assistant, you will return to the screen where you choose whether you want to create your model in Python or OPL, with the Modeling Assistant or in a Python notebook or import an existing model. If you choose to replace your model at this stage, you will overwrite your current model and lose your changes. If you want to keep a copy of your current work in progress, create a new scenario before changing the model.

Duplicate the scenario

To keep a copy of this model, make a copy of this scenario:

1. If the scenario pane is not open, click the Scenarios icon.
2. Click the three dots next to Scenario 1 and select Duplicate.
3. Enter a name for the new scenario, Scenario 2, for example, and click Create. You continue working in Scenario 2.

Complete your model

The constraints are no longer highlighted once you have entered values. The model, however, isn't quite complete. You might want to make sure that your schedule takes into account the order of precedence of tasks so that each activity can only start after those that must precede it. You will add this constraint later.

If your model had more objectives and constraints, you could browse or filter them by using the Find in my objectives and constraints search field.

You can choose to Disable or Remove any one of the objectives or constraints in your model by clicking the 3 vertical dots next to the statement. This menu also enables you to reorganize the order of your statements by moving them up and down and you can also duplicate a statement.

When you have completed your model, or when there are no objectives or constraints still highlighted, you can run it to find a solution that will decide the best optimal schedule based on your model objectives and constraints.

Run your model

You can change the solve time limit for your model in the Build model view by clicking the Settings tab next to the suggestions. For this example, use the default limit. Other parameters can also be set using Run Configuration parameters (see Configuring the run parameters for a scenario in a Decision Optimization experiment for more information).

In Scenario 2, click the Run button. When an initial objective value has been found, a Combined Objective is displayed in a Solve Progress graph. If you want to end this run before the optimal solution is obtained, you can quit by clicking Stop. When the optimal solution has been found the Explore solution view opens.

Your solution

When the run is completed, you can see the results in the Explore solution view. You can also click Engine statistics or Log to see the solution chart and inspect the log files. The first tab in the Explore solution view shows the objective (or objectives if you have several) with its values and weights. The Solution tables tab provides you with the best schedule with the assignment of activities to subcontractors.

You can also download the solution tables as csv files.

If your model had any conflicting constraints, these would be shown in the Conflicts tab with the Relaxations necessary to solve the model.

In the Visualization view, click Gantt to display the solution as a Gantt chart.

Create a new scenario - different model, same data

Although you have solved the model and now have the optimal schedule for your activities with subcontractors assigned, you haven't as yet considered the precedence of activities nor the skill level data of your subcontractors in making the assignments. Scenarios enable you to analyze and compare different models and data.

First, examine a new scenario with an additional constraint:

In the model view, other objectives and constraints are offered in the Suggestions pane. You can add these to your model by clicking them. To see other suggestions that are not listed, start typing in the search field and press enter or the refresh button. You can then browse and add from the displayed propositions.

To add the precedence constraint to your model to ensure that there are no time lags between activities:

1. Duplicate Scenario 2 and call it Scenario 3. Then close the scenario pane.
2. In the Suggestions pane in the model view, type in natural language activity after preceding activities, for example, in the search field and click enter.
3. From the new list of suggestions, click Each house_activity starts after the end of preceding activities to add it to your constraints.

   The new precedence constraint appears in your model formulation.

4. Rerun the model (scenario 3) and look at the new solution. You can compare this to the solution you obtained in scenario 2, when you solved the model without this constraint. To compare solutions, open the Open scenario pane pane and click each scenario. You can also click Gantt in the Visualization view and compare solutions displayed as Gantt charts for each scenario.

1. Duplicate Scenario 3 and call it Scenario 4. Then close the scenario pane.

   To maximize the subcontractors' skill levels in their assignment to activities:

2. In the model view, type overall quality in the suggestions search field to find and add the following objective to your model :

   Maximize overall quality of house_subcontractor-house_activity assignments according to table of assignment values.

   Click the underlined <table of assignment value> and type or select house_expertise.

   Your new objective is now Maximize overall quality of house_subcontractor-house_activity assignments according to house_expertise. Expand the objective and select Activity for the task, Subcontractor for the resource, and Skill level for the value, (table columns) to complete the definition.

   You now have two objectives. You can decide whether the objectives are to be considered equally or with different weightings. You can increase and decrease the weights on each objective by using the adjacent slider. Leave the two sliders at 5 so that your two objectives are equally weighted. You can also add scale factors for the objectives. For this example, leave the scale factors as 1. For more information, see Weights and scale factors displayed in the solution.

3. To ensure that subcontractors only undertake tasks that they are permitted to do, add a new constraint. Type in the suggestions, for example, subcontractor must be one of possible subcontractors. You can also set Display by category to on (a tick is displayed on the switch) and select the filter Assignment to see suggestions related to assignment.
4. From the filtered suggestions, find and add the following constraint: For each house_subcontractor-house_activity assignment, assigned house_subcontractors must be one of Possible Subcontractors of house_activity.
5. Run the model

The new solution now shows a new assignment of subcontractors.

More about the model view

The Suggestions filter

You can also filter the suggestions to find objectives and constraints. Set Display by category to on (a tick is displayed on the switch) which opens a pane for you to select various categories of interest and apply filters to the list of suggestions. The filters enable you to see fewer suggestions. If you click the display icon next to the search field, you can see all possible expressions for the scheduling domain including those that are disabled. Hovering over the information icon for each expression provides you with a description. For disabled terms hovering over the expression itself also gives you an explanation for why it is disabled for this model.

The Settings tab

The Settings tab in the model view lists different scheduling and optimization parameters that can be edited. In this example the default duration unit, the optimization run time and the date/time format are shown. You can specify here a customized date/time format to suit your data.

The Data Schema tab

The Data Schema tab view lists, table by table, all information that the Modeling Assistant has imported and deduced from the input data that is necessary for the scheduling problem to be solved. You can edit certain entries in the schema which will update your model and prompt you to accept the implied model changes or cancel your edits. This can be useful for expert users for data debugging purposes. For example if a column containing an ID has been deduced as numerical, it might be useful to change this to nominal so that it can be used as a primary key.

The Decisions tab

You can make this tab visible by setting the Visualize and edit decisions in the Settings tab to true. The Decisions tab shows you the decision (or intent) that is defined in your model. You can also add custom decisions here. For more information see Defining custom decisions.

Generating a Python notebook from your scenario

If you want to generate a Python notebook from your model created with the Modeling Assistant:

1. If the scenario pane is not open, click the Scenarios icon.
2. Click the three dots next to one of your scenarios and select Generate notebook.
3. Enter a name for your notebook and click Generate.

A Python notebook for this model is created in your Project.

Overview pane

You can view summary information for all your scenarios at a glance in the Overview pane. By selecting a scenario and clicking the three dots, you can perform actions such as duplicate, rename, generate a Python notebook, export the scenario, or save it for deployment, for any selected scenario in this view. See Viewing all scenarios in a Decision Optimization experiment for more information on how to configure this pane.

Visualization view

In the Visualization view you can customize what you want to see displayed from any scenario. You can view your input data, your solution and add notes. For example, for this house tutorial, you can see a Gantt chart for the optimal solution schedule.

You can use table widgets and chart widgets to customize the layout of these views. You can add headers, change background colors and other properties of your notes, tables or charts. You can choose different types of charts such as line charts, bar charts, and so on. You can define how data is aggregated in these charts and use the calculate property to define how to represent certain data values in your charts.

If you select a table or chart widget, a default instance is displayed using some of your input data. To change the content and format of this object, click the pencil icon and edit the widget with either the graphical editor or by editing the json file.

You can use this view to visually compare scenarios.

For more information, see Visualization view in a Decision Optimization experiment.